Skip to content

Restart, disks and skewed clocks - #20

Merged
dhruvl merged 7 commits into
mainfrom
fault-model
Aug 1, 2026
Merged

Restart, disks and skewed clocks#20
dhruvl merged 7 commits into
mainfrom
fault-model

Conversation

@dhruvl

@dhruvl dhruvl commented Aug 1, 2026

Copy link
Copy Markdown
Owner

The fault model grows past the network: machines can die and come back, keep what they wrote, and disagree about what time it is.

  • loop.net.restart(name) / host.restart() make crash a round trip. Restart revives liveness and nothing else — the caller boots the revived machine the way it booted the original. Crash and restart now record trace events, so a workload that crashes a host hashes differently than it did in 0.1.0; workloads that never crash stay byte-identical (checked against the v0.1.0 tag over 50 seeds).
  • host.disk is a per-host mapping that survives crashes and restarts — the place for state a real process would fsync. Crash never touches it, structurally.
  • loop.net.set_clock(host, offset=...) skews what that host's tasks read from loop.time() and where their call_at deadlines land, while durations — sleep, timeout, wait_for, call_later — cost the same true time everywhere, which is what a wrong wall clock actually does. Internal scheduling and trace timestamps stay on the true clock, so zero-skew runs are byte-identical to 0.1.0 and traces from skewed runs remain comparable.
  • One additive jobqueue test drives a worker whose clock runs two leases ahead: renewals hold exactly-once through a job longer than its lease, and a trace-hash comparison shows the app is immune to skew by construction — it exchanges durations, never timestamps. Clock faults only bite code that compares timestamps taken on different machines, and the docs now say so.

Suite green including determinism over the new surface; docs updated (supported-api rows, README, CHANGELOG); no new runtime dependencies.

dhruvl added 7 commits August 1, 2026 14:20
The clock row read as if durations were skewed too, and the restart
notes promised finished cancellations where crash only requests them.
Also record that crashes now leave a trace event, which moves the hash
of any workload that crashes a host.
Cancel in name order so the teardown cannot depend on set iteration,
and check the skew immunity hash on a second seed. The call_at test
kept an assertion its own comment claimed was independent.
@dhruvl
dhruvl merged commit 893d709 into main Aug 1, 2026
4 checks passed
@dhruvl
dhruvl deleted the fault-model branch August 1, 2026 10:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant