ignore: Avoid contention on num_pending

[tavianator]

Previously, every worker would increment the shared num_pending count on
every new work item, and decrement it after finishing them, leading to
lots of contention. Now, we only track the number of workers actively
running, so there is no contention except when workers go to sleep or
wake up.

[tavianator]

Quick benchmark results (updated):

tavianator@tachyon $ hyperfine -w2 fd-{before,after}" -j24 -u '^$' ~/code/bfs/bench/corpus/chromium"
Benchmark 1: fd-before -j24 -u '^$' ~/code/bfs/bench/corpus/chromium
  Time (mean ± σ):     297.6 ms ±  10.9 ms    [User: 3453.3 ms, System: 3379.4 ms]
  Range (min … max):   279.9 ms … 313.3 ms    10 runs
 
Benchmark 2: fd-after -j24 -u '^$' ~/code/bfs/bench/corpus/chromium
  Time (mean ± σ):     231.6 ms ±   6.9 ms    [User: 1568.7 ms, System: 3660.1 ms]
  Range (min … max):   224.6 ms … 248.9 ms    12 runs
 
Summary
  fd-after -j24 -u '^$' ~/code/bfs/bench/corpus/chromium ran
    1.29 ± 0.06 times faster than fd-before -j24 -u '^$' ~/code/bfs/bench/corpus/chromium
tavianator@tachyon $ hyperfine -w2 fd-{before,after}" -j48 -u '^$' ~/code/bfs/bench/corpus/chromium"
Benchmark 1: fd-before -j48 -u '^$' ~/code/bfs/bench/corpus/chromium
  Time (mean ± σ):     226.1 ms ±  17.9 ms    [User: 4597.6 ms, System: 4339.5 ms]
  Range (min … max):   210.0 ms … 260.0 ms    13 runs
 
Benchmark 2: fd-after -j48 -u '^$' ~/code/bfs/bench/corpus/chromium
  Time (mean ± σ):     199.6 ms ±  17.7 ms    [User: 2232.0 ms, System: 5361.9 ms]
  Range (min … max):   185.8 ms … 238.4 ms    15 runs
 
Summary
  fd-after -j48 -u '^$' ~/code/bfs/bench/corpus/chromium ran
    1.13 ± 0.13 times faster than fd-before -j48 -u '^$' ~/code/bfs/bench/corpus/chromium

[BurntSushi]

Thanks! I'll need to think on this one. I've had problems with termination in the past and this changes the logic around that.

[tavianator]

Thanks! I'll need to think on this one. I've had problems with termination in the past and this changes the logic around that.

Understandable! I am reasonably sure of the correctness now that I've strengthened the memory orderings to acquire/release. But it's not a formal proof, and I haven't tried too hard to break it either.

[BurntSushi]

Oh, also, what about the relaxed loads? It looks like they should be fine to me here, but I wonder if Acquire/Release are necessary. I think that's probably why I used SeqCst originally (because I didn't think through what was actually required).

[tavianator]

Oh, also, what about the relaxed loads? It looks like they should be fine to me here, but I wonder if Acquire/Release are necessary. I think that's probably why I used SeqCst originally (because I didn't think through what was actually required).

I think it was still correct with relaxed accesses due to other orderings imposed by the deque implementation, but it's easier to argue correctness with acquire/release and they're not performance-critical any more so I switched them.

[BurntSushi]

OK, I've tried this out:

1. I think I buy that this is a correct change.
2. I have actually tried it on a number of corpora and everything looks OK.
3. I haven't been able to observe any reliable performance difference. Nevertheless, I buy the argument here.

Thank you!!!