added moodycamel/concurrentqueue as the default task queue provider f…

[glglwty]

…or fastrun. Fixed unnecessary seq_cst sync in task::task_state. added benchmark for task_queue. Set default sim_net delay to 0.

about the queue

moodycamel/concurrentqueue is a fast MPMC queue, released under BSD licence. It is much faster than naive implementations, including boost and TBB implementations. Its implementation is complicated and I don't think there is a way to hand-craft a queue that is comparable to it.

performance change

I tested the performance of hpc_priority_task_queue and the newly added hpc_concurrent_task_queue using core.task_queue_perf_test. Here's their performance:

hpc_concurrent_task_queue

inter-thread flooding test:throughput = 6672467
self-flooding test:throughput = 7226603
inter-thread blocking test:throughput = 1046366
self-iterating test:throughput = 4041782
tick-tock test:throughput = 2700396

hpc_priority_task_queue

inter-thread flooding test:throughput = 5152068
self-flooding test:throughput = 7956709
inter-thread blocking test:throughput = 100560
self-iterating test:throughput = 6062134
tick-tock test:throughput = 266755

We can see that hpc_concurrent_task_queue is much faster under contention or partially-idle
workloads.

discussion about performance

The performance of our task_queues is not ideal. I did profiling on the benchmarks and the current most prominent overhead lies within the free procedure of tasks. Maybe memory pool of frequently used objects is needed.

discussion about atomic variables

It seems that many of us aren't aware of the proper usage of std::atomic and memory_order. Here is some rule of thumb:

1. the default memory order of atomic variables is std::memory_order_seq_cst. It has high overhead (~100 cycles on x86) and is hardly necessary. Actually a well-behaved spinlock does not need seq_cst synchronization.
2. Acquire-release pairs is (and should be!) widely used to ensure safety.
3. Consume-release pairs might be a faster version of acquire-release if you are aware of data dependency around atomic operations.

I recommend this article for reference: https://preshing.com/20140709/the-purpose-of-memory_order_consume-in-cpp11/

broken assumptions

default sim_net delay is set to 0 to avoid confusion
the default task_queue factory is hpc_concurrent_task_queue now

misc

@imzhenyu you can update imzhenyu/concurrentqueue from upstream

[imzhenyu]

Very nice summary. Thanks, Tianyi. Now that we are working on performance tuning, it is strongly recommended that everyone reads the article that Tianyi suggests above. Meanwhile, let's write down all the items we have optimized. Later on we can come up a list to avoid making the same mistakes in the future (I myself made certain mistakes). Furthermore, we may discuss setting up some performance regression test next.

[imzhenyu]

imzhenyu/concurrentqueue updated as suggested.

[imzhenyu]

cannot imagine this bug exists here for so long and we did not find it:)

[qinzuoyan]

It's really a great work~

[imzhenyu]

There seems certain overhead when enqueue and dequeue are in the same thread (33% slowdown for self-iterating test). It is possible we can have a local queue for local enqueue operations when the queue is private (when the thread pool is partitioned).