Skip to content

perf(waitgroup): streamline completion for participants and observers - #421

Open
nanami-0713 wants to merge 4 commits into
apache:mainfrom
nanami-0713:perf/waitgroup-streamline-completion
Open

nanami-0713 wants to merge 4 commits into
apache:mainfrom
nanami-0713:perf/waitgroup-streamline-completion

Conversation

@nanami-0713

Copy link
Copy Markdown
Contributor

Summary

Continues #395 on top of current main (629e073). The four commits rebase cleanly: git range-diff shows three commits identical and only the CHANGELOG entry moving to the current Unreleased section. Fixes #397.

The change, as sketched in #395:

  • Participants share one allocation reference, so cloning and dropping a worker updates only the participant count instead of the shared pointer.
  • The first pending observer registers an inline waker; additional observers use the existing cancellable waker set.
  • The registry lock becomes an RwLock, whose uncontended state stays inline on Darwin, where locking a standard mutex allocates a pthread mutex on first use.

Measured on Apple M5 (10 cores), macOS 27.0.1, rustc 1.99.1 stable, with cargo x bench --bench ecosystem -- waitgroup --repeat 10:

Scenario main 629e073 this PR waitgroup 0.1.2
complete-waiter 175 ms 52 ms 61 ms
complete-4-workers 170 ms 68 ms 64 ms
drain-4-observers — 129 ms 502 ms (shared drain)

A #[global_allocator] probe counts three allocations (208 B) per complete-waiter round on main — the state allocation, Darwin's pthread-mutex boxing on the first lock, and the first Arena<Waker> insert — and one allocation per round on this PR: 56 B, 16 B above waitgroup's 40 B, covering the inline first-waker slot.

cargo x test (including the 15 waitgroup integration tests) and cargo x lint pass on the rebased branch.

Comment on lines +153 to +156
#[inline(never)]
fn finish(&mut self) {
self.waiters.wake_all();
self.release_observer();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could a panic in wake_all() prevent release_observer() from running and end up leaking the state? Maybe we should ensure cleanup still happens if a waker panics.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Walking the path: Waiters::wake_all detaches the slots under the write lock and wakes after unlocking, so a panicking wake callback runs with no internal lock held. If one still panics:

  • The observers after the panicking one never receive their wake. That is inherent to a panic mid-notification and predates this change — the previous implementation also mem::take-ed the waker collection under the lock and called wake_all outside it, so a panic in the same place stranded the remaining observers identically.
  • What this change does alter is reclamation: with count-based ownership the panicking path skips release_observer, so the state allocation is not reclaimed while unwinding, where the previous Arc-based implementation would have freed it.

That difference lives entirely on a path the repository's waker contract excludes: executor waker operations, including waking and dropping, are expected not to panic, and primitives deliberately add no custom panic recovery (AGENTS.md "Waker Contract", decision #257). A drop guard in finish that releases the observer count on unwind would be exactly that recovery machinery, and the contract's review guidance is to not require it absent an explicit local contract.

@Kaileshwar16

Copy link
Copy Markdown
Contributor

i also think earlier PR showed a small performance regression on Linux in the four-observer case. It would be worth rerunning the benchmark to check whether thats still present.

@nanami-0713

Copy link
Copy Markdown
Contributor Author

On the four-observer Linux point: the only Linux four-observer data I can find is the arm64 Linux VM control documented in #395 — a standalone driver repeating the same four-observer lifecycle, with ten-run medians of 159 ms on main vs 164 ms on the revised head 540858b (the content of this PR), about 3% above main.

I reran the comparison on hardware I have (Apple M5, 10 cores, macOS 27.0.1, rustc 1.99.1 stable), back-to-back in one session, --repeat 10. Base is 629e073 with this branch's benchmark commit cherry-picked on top; candidate is this branch.

Scenario Rounds main + bench this PR
drain-4-observers 2,500,000 191 ms (190..192) 129 ms (128..130)
complete-4-workers 2,600,000 174 ms (173..179) 67 ms (67..67)

So on arm64 macOS the revised implementation is clearly ahead on the four-observer drain, matching #395's own M4 Max numbers for the same comparison (294 → 230/221 ms); the small deficit exists only in the Linux VM control. I don't have Linux hardware to rerun that control. The scenario is now in-tree, so a Linux check is one command away for anyone who has the hardware:

cargo x bench --bench ecosystem -- waitgroup/drain-4-observers --repeat 10

Happy to fold in a Linux datapoint if someone can run it.

@Kaileshwar16

Copy link
Copy Markdown
Contributor

@nanami-0713 totaly missed it, thanks for clarifying : )

@Kaileshwar16

Copy link
Copy Markdown
Contributor

Ran the four-observer benchmark on native x86_64 Linux (Intel), using 2.5M rounds and --repeat 10.

  • First run: main 597 ms, PR 576 ms (~3.5% faster)
  • Reverse-order run: main 631 ms, PR 582 ms (~7.8% faster)

Used 629e073 with the benchmark commit cherry-picked for the baseline. The PR was consistently faster in both comparisons on my machine, so I didnt observe a regression on x86_64 Linux.

Hope this helps with the Linux benchmark comparison <3

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bring WaitGroup completion to parity with waitgroup

3 participants