[Idea] Hiding plugin instance at library mode

[nokute78]

Now, flb_output_new and flb_input_new return a pointer of plugin instance.
Would you move them to flb_ctx_t to hide from user ?

Like this.

struct flb_lib_ctx {
    struct mk_event_loop *event_loop;
    struct mk_event *event_channel;
    struct flb_config *config;
    struct flb_input_instance *in;
    struct flb_output_instance *out;
};

User specify plugin instance with plugin id (like fd).

   int in_id = 0;
   ...
   in_id = flb_input_new(ctx, "mem", NULL); /* In this function, generate a instance and set  ctx->in[in_id] to the instance */
   flb_input_set(in_id, ctx, "tag", "test", NULL); /* set ctx->in[in_id]->properties to "tag" */
   ...
   flb_destroy(ctx); /* release all of ctx->in */

Pros:

• User takes care of only flb_ctx_t. User doesn't need to release plugin instances. Engine will do.
• Engine can control stop and release timing of instances in flb_stop or flb_destroy. (Now, it depends on user code.)
• User also can specify plugin with plugin id.

Cons:

• Lose compatibility.

How about this?

[edsiper]

the main reason of getting a reference to the input/output instance, is to be able to set properties or any other kind of future modification.

Note that when the end-user call flb_stop(ctx), the engine receives the message and it start releasing resources, internally it does an exit for all input/output instances associated to the main context.

The user never need to do an explicit release of the instances, all this happens in the background.

[nokute78]

Thank you for your comment.
I noticed that flb_ctx_t already contains plugin instances.

the main reason of getting a reference to the input/output instance, is to be able to set properties or any other kind of future modification.

So, why should we expose a plugin instance ?
User passes these instances to flb_*put_set as identifier.
I think structure of instance is not necessarily need, only identifier value is needed.

The user never need to do an explicit release of the instances, all this happens in the background.

I understand. But I think it is odd from user side.
Because flb_stop(ctx) doesn't contain plugin instances at first sight.
However this function influences plugin instances.
This means fluent-bit treats these instances as part of context implicitly.

So , I proposed that plugin instances should be hid from user and modify API.
flb_input_set( input_ins, ... ) -> flb_input_set( input_id, ctx, ... )

[edsiper]

I am thinking about this, no strong opinion yet, once I finish buffering/scheduling stuff I will focus on API and share more updates.

[nokute78]

I see, I'm waiting.
Thanks!

[edsiper]

I am reviewing this. I think it make sense to have an integer as identifier, thanks for pointing that.

[edsiper]

@nokute78 I have pushed the changes into api_ffd branch.

note: the remaining part are tests.

[edsiper]

I just migrated all tests/examples, changes rebased on master.

If is there anything pending or something not working as suggested please add your comments to this tickets.

[nokute78]

Amazing!! That's what I imagined!
Thanks.