sync: Add disarm to mpsc::Sender

[jonhoo]

Fixes #898.

Motivation

Basically #898. Also needed for tower-rs/tower#408.

Since poll_ready takes up one of the finite number of slots in a bounded channel, callers need to send an item shortly after poll_ready succeeds. If they do not, idle senders may take up all the slots of the channel, and prevent active senders from getting any requests through. Consider this code that forwards from one channel to another:

loop {
  ready!(tx.poll_ready(cx))?;
  if let Some(item) = ready!(rx.poll_recv(cx)) {
    tx.try_send(item)?;
  } else {
    break;
  }
}

If many such forwarders exist, and they all forward into a single (cloned) Sender, then any number of forwarders may be waiting for rx.poll_recv at the same time. While they do, they are effectively each reducing the channel's capacity by 1. If enough of these forwarders are idle, forwarders whose rx do have elements will be unable to find a spot for them through poll_ready, and the system will deadlock.

Solution

disarm solves this problem by allowing you to give up the reserved slot if you find that you have to block. We can then fix the code above by writing:

loop {
  ready!(tx.poll_ready(cx))?;
  let item = rx.poll_recv(cx);
  if let Poll::Ready(Ok(_)) = item {
  } else {
    // give up our send slot, we won't need it for a while
    tx.disarm();
  }
  if let Some(item) = ready!(item) {
    tx.try_send(item)?;
  } else {
    break;
  }
}

[LucioFranco]

LGTM, just that one example sticks out.

[carllerche]

I would recommend removing doc hidden. The APIs are public anyway and documented.