feat(memtrack): pause producers under ring pressure - #543
not-matthias wants to merge 10 commits into
Conversation
Merging this PR will not alter performance
|
15e7943 to
89556be
Compare
|
31b6da1 to
0431cd9
Compare
0431cd9 to
58f8c9b
Compare
Share one poll interval across the event, stack and attach pollers and lower it from 10ms to 1ms so bursts drain before the rings fill.
A full allocation-stack ring loses stack records the same way a full event ring loses events, so a run that overflowed it must fail the same incompleteness check.
The FNV lanes lived on the BPF stack. Large kprobe-family programs may spill that to per-CPU storage, which a nested uprobe on the same CPU can overwrite mid-capture, corrupting the hash. Accumulate the lanes in the not-yet-submitted ring record instead, which is private to this reservation.
58f8c9b to
cba9475
Compare
This comment has been minimized.
This comment has been minimized.
After every event or stack submission, BPF checks the ring's fill level. Once it is 75% full, the writing tracked process is recorded in `pressure_stopped` and gets SIGSTOP, so processes that don't write keep running. The event and stack pollers resume every recorded process once a poll leaves their ring empty, and resume everything still recorded on shutdown. A tracked process that writes to a nearly full ring after the pollers are gone stays stopped. A process can be stopped both for ring pressure and for an allocator attach request, and SIGSTOP is not counted. The exec-mapping watcher therefore records its stops in `attach_stopped`. Each side deletes its own entry before checking the other's, so the process resumes only once both are done with it. `RingBufferPoller::drain` no longer acknowledges a consume that stopped at an uncommitted reservation, and `wait_all_stopped` treats exited threads as stopped. The event and stack poll interval is configurable through the `poll_interval_ms` tracker option (env `CODSPEED_MEMTRACK_POLL_INTERVAL_MS`, default 1ms), which lets the event ring cross its watermark on demand. The attach poller keeps its fixed interval.
Add `alloc_storm` (threads) and `alloc_storm_procs` (forked processes) fixtures and pressure tests that run them with a 10s poll interval and assert that no events are dropped. The multi-process test checks that every writing process is stopped and resumed on its own.
On glibc >= 2.42 the per-thread tcache is initialized lazily. A thread's first small free() whose tcache is still inactive goes through tcache_free_init(), which tail-calls __libc_free() again, so the free uprobe fires twice for one call. Whether a thread reaches that path depends on arena assignment, i.e. scheduling, so the Free count of the same workload varies between runs. for_each_variant compared raw Free counts between the Legacy and Token runs, which made test_thread_dlopen flaky on ubuntu-26.04-arm (glibc 2.43). GLIBC_TUNABLES (tcache_count=0, tcache_max=0) does not avoid the re-entry. event_profile now replays events in timestamp order and counts a Free only when it releases an allocation still live in that run, which drops the duplicate hit as well as frees of memory allocated before tracking.
The stop maps kept a process's entry after it exited, so a later release could send SIGCONT to an unrelated process that reused the pid. The exit handler now deletes a process from `pressure_stopped` and `attach_stopped`, and a release resumes a process only if it removed its own entry. The exec-mapping watcher stops a process only once it is recorded, like the pressure check, so every stop has an entry to release. Both maps move to a shared header so the exit handler can reach them.
d57345a to
c023567
Compare
| __u8 marker = 1; | ||
| if (bpf_map_update_elem(&pressure_stopped, ¤t_tgid, &marker, BPF_ANY) != 0) { | ||
| return; | ||
| } |
There was a problem hiding this comment.
After the BPF code records a pressure hold, the independent poller can observe an empty ring, delete that record, and send SIGCONT before bpf_send_signal(SIGSTOP) runs. The later SIGSTOP then suspends the producer with no remaining record that can resume it, causing the tracked workload and capture to hang.
Knowledge Base Used: eBPF memory tracker
Prompt To Fix With AI
This is a comment left during a code review.
Path: crates/memtrack/src/ebpf/c/utils/pressure.bpf.h
Line: 34-37
Comment:
**Pressure Stop Can Be Lost**
After the BPF code records a pressure hold, the independent poller can observe an empty ring, delete that record, and send `SIGCONT` before `bpf_send_signal(SIGSTOP)` runs. The later `SIGSTOP` then suspends the producer with no remaining record that can resume it, causing the tracked workload and capture to hang.
**Knowledge Base Used:** [eBPF memory tracker](https://app.greptile.com/codspeed/-/custom-context/knowledge-base/codspeedhq/codspeed/-/docs/ebpf-memory-tracker.md)
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.
Adds opt-in, best-effort backpressure for memtrack's BPF ring buffers.
When a ring crosses its watermark, BPF latches the episode and stops the winning tracked producer. A userspace coordinator then stops the owned process tree using pidfds, drains the event, stack, mapping, and attach pipelines, clears the latch, and resumes the tree. Finite timeouts terminate the tracked tree and fail the capture;
0remains the default and preserves drop detection, whileinfwaits until recovery. Detected ring loss still fails the capture.The change also:
This PR is stacked on #522.
Verification
cargo fmt --all -- --checkcargo clippy --release -p memtrack -p codspeed-runner -- -D warnings