-
-
Notifications
You must be signed in to change notification settings - Fork 2.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
sync: reduce contention in Notify
#5503
Conversation
This PR is rather large. Will take me a bit until I can review it. |
Sure, no problem. I've pushed an update which slightly reduces the size of this change. I can write down some key points to make the review easier. Adding the atomic variable is quite straightforward, it boils down to adding atomic stores to Previously, a Waiters for the batch semaphore also use an atomic field for tracking the number of permits, and the code there is organized similarly to this PR. If this PR gets merged, I would like to also investigate if similar optimization can be applied in the IO driver. |
tokio/src/sync/notify.rs
Outdated
/// `true` if the notification has been assigned to this waiter. | ||
notified: Option<NotificationType>, | ||
/// Notification for this waiter. It is an enum and uses | ||
/// `NOTIFICATION_{NONE, ONE, ALL}` values. | ||
notification: AtomicUsize, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the code is ok, but the safety invariants invovled with this atomic should be documented more clearly.
* use wrapper type for the atomic notification field * fix safety commments * document the fact that the notification field is used to protect `waker` field
I've added some comments to better document the safety invariants. I've also moved the atomic variable into a wrapper type to use an enum instead of constants. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I reviewed this again, and it LGTM.
This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [tokio](https://tokio.rs) ([source](https://github.com/tokio-rs/tokio)) | dependencies | minor | `1.27.0` -> `1.28.0` | | [tokio](https://tokio.rs) ([source](https://github.com/tokio-rs/tokio)) | dev-dependencies | minor | `1.27.0` -> `1.28.0` | --- ### Release Notes <details> <summary>tokio-rs/tokio</summary> ### [`v1.28.0`](https://github.com/tokio-rs/tokio/releases/tag/tokio-1.28.0): Tokio v1.28.0 [Compare Source](tokio-rs/tokio@tokio-1.27.0...tokio-1.28.0) ##### 1.28.0 (April 25th, 2023) ##### Added - io: add `AsyncFd::async_io` ([#​5542]) - io: impl BufMut for ReadBuf ([#​5590]) - net: add `recv_buf` for `UdpSocket` and `UnixDatagram` ([#​5583]) - sync: add `OwnedSemaphorePermit::semaphore` ([#​5618]) - sync: add `same_channel` to broadcast channel ([#​5607]) - sync: add `watch::Receiver::wait_for` ([#​5611]) - task: add `JoinSet::spawn_blocking` and `JoinSet::spawn_blocking_on` ([#​5612]) ##### Changed - deps: update windows-sys to 0.48 ([#​5591]) - io: make `read_to_end` not grow unnecessarily ([#​5610]) - macros: make entrypoints more efficient ([#​5621]) - sync: improve Debug impl for `RwLock` ([#​5647]) - sync: reduce contention in `Notify` ([#​5503]) ##### Fixed - net: support `get_peer_cred` on AIX ([#​5065]) - sync: avoid deadlocks in `broadcast` with custom wakers ([#​5578]) ##### Documented - sync: fix typo in `Semaphore::MAX_PERMITS` ([#​5645]) - sync: fix typo in `tokio::sync::watch::Sender` docs ([#​5587]) [#​5065]: tokio-rs/tokio#5065 [#​5503]: tokio-rs/tokio#5503 [#​5542]: tokio-rs/tokio#5542 [#​5578]: tokio-rs/tokio#5578 [#​5583]: tokio-rs/tokio#5583 [#​5587]: tokio-rs/tokio#5587 [#​5590]: tokio-rs/tokio#5590 [#​5591]: tokio-rs/tokio#5591 [#​5607]: tokio-rs/tokio#5607 [#​5610]: tokio-rs/tokio#5610 [#​5611]: tokio-rs/tokio#5611 [#​5612]: tokio-rs/tokio#5612 [#​5618]: tokio-rs/tokio#5618 [#​5621]: tokio-rs/tokio#5621 [#​5645]: tokio-rs/tokio#5645 [#​5647]: tokio-rs/tokio#5647 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about these updates again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNS42My4xIiwidXBkYXRlZEluVmVyIjoiMzUuNjQuMCJ9--> Co-authored-by: cabr2-bot <[email protected]> Reviewed-on: https://codeberg.org/Calciumdibromid/CaBr2/pulls/1875 Reviewed-by: crapStone <[email protected]> Co-authored-by: Calciumdibromid Bot <[email protected]> Co-committed-by: Calciumdibromid Bot <[email protected]>
This PR replaces the notification field used by waiters in
Notify
with an atomic variable to reduce lock contention when multiple pendingNotified
futures are completing.Motivation
The typical lifecycle of a
Notified
future looks like this:notify_one
ornotify_waiters
call and is removed from the waiting list.wake()
on the future's waker.Ready
.Currently, step 4 requires acquiring the lock to read the notification field and the lock is released immediately after. Using an atomic variable here should significantly reduce lock contention, especially when multiple futures get notified at once by a
notify_waiters
call.Benchmarks
Simple benchmarks on my machine (Linux 6.1.11, 4-core x86_64 developer laptop) show improved performance by 15-35%.
Click to expand results
master (
c894069
)with atomic notification:
With many contending waiters (>500) benchmarks show degraded performance, but I did some profiling and the bottleneck seems to be somewhere in the scheduler.