End-to-end object size information passthrough #90

atumanov · 2016-12-06T01:07:16Z

pass through object size information from Plasma store to Plasma manager to Redis.
add object subscribe methods for schedulers to listen to object information.

pcmoritz · 2016-12-08T06:55:05Z

src/global_scheduler/global_scheduler.c

@@ -71,6 +72,24 @@ void process_new_db_client(db_client_id db_client_id,
 }
 }

+/* object table subscribe callback


Can you bring these comments in the same format as the other format strings?

first line: /**
second line: * with a one line description of the fct
third line: *
then the args
then @return Void.

pcmoritz · 2016-12-08T06:57:48Z

src/plasma/plasma_client.c

@@ -571,6 +571,12 @@ int plasma_wait(plasma_connection *conn,
 plasma_free_request(req);
 int64_t return_size = plasma_reply_size(num_returns);
 plasma_reply *reply = malloc(return_size);
+ int rv = plasma_receive_reply(conn->manager_conn, return_size, reply);
+ if (rv < 0) {
+ fprintf(stderr, "plasma_wait: failed with rv=%d, errno=%d, errno=%s\n",


use checkm (with a message) here, it will automatically print the errno

pcmoritz · 2016-12-08T06:59:32Z

src/plasma/plasma.h

+typedef struct {
+ object_id obj_id;
+ int64_t data_size;
+} objectid_notification;


can we rename this to object_id_notification to be consistent with the rest of the code?

pcmoritz · 2016-12-08T06:59:53Z

src/plasma/plasma_manager.c

 retry_info retry = {
 .num_retries = NUM_RETRIES,
 .timeout = MANAGER_TIMEOUT,
 .fail_callback = NULL,
 };
 /* Read the notification from Plasma. */
- int error = read_bytes(client_sock, (uint8_t *) &obj_id, sizeof(obj_id));
+ objectid_notification objid_notification;


consider obj_id_notification here

pcmoritz · 2016-12-08T07:00:57Z

src/plasma/plasma_manager.c

- entry->object_id = obj_id;
- HASH_ADD(hh, state->local_available_objects, object_id, sizeof(object_id),
- entry);
+ memcpy(&entry->object_id, &objid_notification.obj_id,


why not entry->object_id = obj_id_notification.obj_id?

pcmoritz · 2016-12-08T07:04:19Z

src/plasma/plasma_store.c

@@ -400,10 +400,21 @@ void send_notifications(event_loop *loop,
 for (int i = 0; i < utarray_len(queue->object_ids); ++i) {
 object_id *obj_id = (object_id *) utarray_eltptr(queue->object_ids, i);
 /* Attempt to send a notification about this object ID. */
- int nbytes = send(client_sock, (char const *) obj_id, sizeof(*obj_id), 0);
+ object_table_entry *entry; /* Points to an element from a hash table. */


make comment at the end of the line more precise

pcmoritz · 2016-12-08T07:05:26Z

src/plasma/plasma_store.c

+ sizeof(object_id), entry);
+ /* Populate and send object id notification. */
+ objid_notification.data_size = entry->info.data_size;
+ memcpy(&objid_notification.obj_id, obj_id, sizeof(*obj_id));


why not obj_id_notifiaction.obj_id = obj_id?

pcmoritz · 2016-12-08T07:05:44Z

src/plasma/plasma_store.c

@@ -415,6 +426,9 @@ void send_notifications(event_loop *loop,
 send_notifications, plasma_state);
 break;
 } else {
+ fprintf(stderr, "send_notifications:send failed with rv=%d, errno=%d, errno=%s\n",


atumanov force-pushed the object_size branch 2 times, most recently from 57b88e1 to bd84c69 Compare December 8, 2016 01:23

object size passthrough -- squashed commit

566293c

atumanov force-pushed the object_size branch from bd84c69 to 566293c Compare December 8, 2016 01:34

atumanov added 3 commits December 7, 2016 17:39

fixing post-rebase issues

ffea307

object table: handling multiple asynchronous requests in callbacks

5893104

use context for all async redis commands except subscribe

cb0b261

pcmoritz reviewed Dec 8, 2016

View reviewed changes

atumanov added 5 commits December 7, 2016 23:05

addressing Philipp's comments

387a27f

addressing Philipp's comments

96b835c

adding object info table subscribe test

6258248

more polish on the code

d74f0e6

clang-format...

1d42693

atumanov changed the title ~~[WIP] end-to-end object size information passthrough~~ End-to-end object size information passthrough Dec 8, 2016

pcmoritz mentioned this pull request Dec 9, 2016

End-to-end object size information passthrough #105

Merged

pcmoritz closed this Dec 9, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

End-to-end object size information passthrough #90

End-to-end object size information passthrough #90

atumanov commented Dec 6, 2016

pcmoritz Dec 8, 2016 •

edited

Loading

pcmoritz Dec 8, 2016

pcmoritz Dec 8, 2016

pcmoritz Dec 8, 2016

pcmoritz Dec 8, 2016

pcmoritz Dec 8, 2016

pcmoritz Dec 8, 2016

pcmoritz Dec 8, 2016

End-to-end object size information passthrough #90

End-to-end object size information passthrough #90

Conversation

atumanov commented Dec 6, 2016

pcmoritz Dec 8, 2016 • edited Loading

Choose a reason for hiding this comment

pcmoritz Dec 8, 2016

Choose a reason for hiding this comment

pcmoritz Dec 8, 2016

Choose a reason for hiding this comment

pcmoritz Dec 8, 2016

Choose a reason for hiding this comment

pcmoritz Dec 8, 2016

Choose a reason for hiding this comment

pcmoritz Dec 8, 2016

Choose a reason for hiding this comment

pcmoritz Dec 8, 2016

Choose a reason for hiding this comment

pcmoritz Dec 8, 2016

Choose a reason for hiding this comment

pcmoritz Dec 8, 2016 •

edited

Loading