flow.async: In ..._task_loop, add a way to get number of pending jobs in the thread pool job queue.

[ygoldfeld]

In flow::async::*_task_loop, add a way to get number of pending jobs in the thread pool job queue.

[ygoldfeld]

I did not file the original ticket in pre-open-source days but added this comment back then:

That's actually quite a topic I have been thinking about, which is referenced in , regarding better balancing in Segregated_thread_task_loop (note: Cross_... uses boost.asio which balances it itself) and in the optional per_thread_ops() feature of the Concurrent_task_loop hierarchy. The job queue size is helpful for that and other things. It's quite a topic. I'll need to think a lot about this; it's complicated.

--

The link talks about the following Host_resolver class being used elsewhere. For context including it here:

---snip---

Currently Host_resolver starts a manually configured # of threads (a quite conservative high number, 500 default). That's fine - no reason to think it's insufficient (although there may be overhead in so many threads, not something we're deeply worried about, but it's out there).

It would be better to automatically start threads as needed. Minimally:

• Add flow::Concurrent_task_loop (and therefore its 2 impls, especially Cross_thread_task_loop in this case) ability to change number of threads. Least entropy-filled way to do it:
  • Add API to change # of threads, currently set in each impl class's ctor. Actually, this might be better to keep out of the interface and just make it specific to each impl class. Can just be a setter in each class; and make it only allowed while stopped (before start(), after ctor or stop()).
  • Then when start() is executed, it starts the appropriate # of threads.
  • Reminder, make a more ambitious ticket for Flow to track work demand/outstanding work in a generic way, which would allow for:
    • signaling user when demand seems high (so they can, e.g., stop(), change # of threads, start());
    • Segregated_thread_task_loop, and for the pre-made-strands feature of Cross_thread_task_loop, to be able to cleverly send work to the least loaded thread/pre-made strand.
      • But at least add round-robin, which is trivial, instead of the current quick-and-dirty random selection.
• In Host_resolver, keep track of outsanding resolution requests that haven't yet started. Perhaps increase thread count when this goes beyond some configurable threshold.
  • Decreasing the # of threads, IMO, is not critical. But increasing is essential in this scheme.
  • Make this configurable to be on/off.
  • The existing 2 knobs for # threads can remain there, if reduced in importance, for the initial thread count.

---end-snip---

Revisit this with original reporter (Arun Poruri), in case there are more thoughts nowadays.