Skip to content

REPRO: cache-ON Bazel Native stability (5+ no-op pushes) - #11

Closed
b7r6 wants to merge 19 commits into
mainfrom
ci/repro-cache-stability
Closed

b7r6 wants to merge 19 commits into
mainfrom
ci/repro-cache-stability

Conversation

@b7r6

@b7r6 b7r6 commented Oct 2, 2026

Copy link
Copy Markdown
Owner

Reproducibility demo. Each trivial no-op push re-triggers the cache-ON Bazel Native lane to show the ~5-6m wall-clock is stable, not a warm-cache fluke. Measurement-only; do not merge.

rohnnyjoy and others added 19 commits September 30, 2026 21:27
…hina#2826)

should_timeout_operation ended an Executing action once Action.timeout
had passed since it was assigned. The worker enforces the same timeout,
but its clock starts when the command starts, after input fetch, so the
scheduler always fired first. The action was requeued onto the worker
still running it, which refused with AlreadyExists; the retries ran out,
and when the worker's own DEADLINE_EXCEEDED result arrived it was
treated as a stray and the worker was disconnected, taking every other
action on it down too.

The scheduler now enforces Action.timeout in every liveness case, at
Action.timeout plus no_event_action_timeout, measured from the later of
the assignment and the worker's last update on the action. A live
worker reports first, so no liveness special-casing is needed, and the
scheduler still ends an action whose worker never reports: a hung input
fetch, an unkillable child, a broken timeout_handled_externally wrapper,
or an orphan no instance reaps.

The client's DEADLINE_EXCEEDED message now names the deadline that
fired, instead of always citing no_event_action_timeout.

This is the interim: the scheduler cannot observe when the command
starts. TraceMachina#2827 tracks the worker reporting command start as an operation
transition, so the deadline can be anchored to that event.

Co-authored-by: Johnny <[email protected]>
…store, and a missing input names its digest so the client re-uploads it (TraceMachina#2855)
…the worker evicted and idle (TraceMachina#2856)

* Worker: no scheduler call awaited in the run loop, a deadline on keepalives and acknowledgements that reconnects, and HTTP/2 keepalive on the scheduler channel

* Worker: the GoingAway calls on shutdown carry the same deadline

* Worker: the layout CI's rustfmt asks for

* Keep the scheduler stream flowing: ordered processing off the reader, no deadlines on stream messages

* Qualify Code in the worker API server test

* Box the action future before the launch closure

* Box the action chain inside the action closure

* Raise the binary's recursion limit for the worker's action future

* Use try_update for the filesystem store generation counter
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
* chore(deps): update rust crate serial_test to v4
* Update MODULE.bazel.lock
The setup-nativelink-cloud action already ships a committed, world-readable
cache_read-only key (cred_kind=public) that serves fork/untrusted PRs with
no secrets, no uploads (--remote_upload_local_results=false), and no BES.
Default ci-mode to 'cache' when the repo variable is unset so the Bazel
Native test lanes actually use it; an explicit 'off' still disables, and
trusted secret-fed lanes are unchanged. Config-only: no new cache feature,
no fleet, no secrets.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Co-Authored-By: Claude Opus 4.8 <[email protected]>
@github-actions

github-actions Bot commented Oct 2, 2026

Copy link
Copy Markdown

Some of the pull request description still needs filling in:

  • What and why is missing. Please keep the template's headings.
  • How was this verified? is missing. Please keep the template's headings.
  • Risk is missing. Please keep the template's headings.
  • AI assistance is missing. Please keep the template's headings.

Edit the description and this check re-runs on its own. The sections exist because they are the parts a reviewer cannot get from the diff: why the change is needed, how you know it works, what breaks if it is wrong, and which AI tools helped ("None" is a complete answer).

@b7r6

b7r6 commented Oct 3, 2026

Copy link
Copy Markdown
Owner Author

CI-spike measurement/experiment scratch — closing to keep the queue clean. Superseded by the single clean deliverable #25; branch retained if ever needed.

@b7r6 b7r6 closed this Oct 3, 2026
@b7r6
b7r6 deleted the ci/repro-cache-stability branch October 15, 2026 11:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants