Repository navigation
MIT-680: pair a test pass with matched work on the hook's clock - #176
Merged
Merged
Conversation
matched_at is this machine's clock and a run's time was the store's (clock_timestamp()), so a store clock behind this one dropped the pass's credit and left the outcome unknown. Postgres in Docker Desktop runs about a millisecond behind the Mac, which failed the hosted end-to-end test once. Hooks now stamp each test run with their own time, sent as test_run_at beside test_run (older servers check test_run's keys exactly), and pairing compares that with matched_at. Runs from older hooks keep the store's time. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Qc1CQTb8r67J32HpkhS43y
test_run_at kept the client's offset. Python accepts offsets with fractional seconds (+05:00:30.5) that Postgres can't cast to timestamptz, so a stored one would fail every later Stop for the task. Normalize to UTC before storing, and refuse times that overflow when shifted to UTC at the year edges (0001-01-01T00:00:00+01:00) with a 400. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01Qc1CQTb8r67J32HpkhS43y
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
hook_client.on_tool_use).RemoteBackendsends it astest_run_atbesidetest_run, because older servers accept exactly{command, exit_code}insidetest_run. The local backend stores it directly.test_run_at(an ISO time with a UTC offset) and stores it asatin thetest_runevent (web/hooks_api.py).store.task_events.test_runsreads it asObservedRun.client_at, and_follows_matched_work(service/hooks.py) comparesmatched_atwithclient_at. Runs from older hooks don't haveclient_at, so they fall back to the store's clock, which is how it worked before.domain/outcomes.py,store/task_events.pyandweb/hooks_api.pycarry the new field, andtest_hook_observe.py'seventshelper now pops (and checks) the run'satbefore comparing.Why
Ticket:
MIT-680. Root cause: a run's time was the store's clock (task_events.created_at,clock_timestamp()), butmatched_atcomes from the hook's spool, which uses this machine's clock. Pairing (decision 0036) credits a pass only ifrun.at > matched_at. When the store's clock is behind this machine's by more than the gap between the matched edit and the test run, the pass is dropped and the outcome staysunknown. Measured here: Postgres in Docker Desktop reads about 1 ms behind the Mac. In the end-to-end test that gap is only a few milliseconds, so it failed occasionally. Decision 0036 already listed this as a known limit and named this fix. A hosted server whose clock is seconds off would hit it all the time.It is not pollution between tests: the failing run's log showed both tool-use calls posted, and the Bash call posts only when a test run was detected. With the run recorded and
used = yes, clock pairing is the only path that leavesunknown.How it was tested
test_a_pass_after_the_fix_counts_when_this_machines_clock_runs_ahead: this machine's clock is set a minute ahead (spool and hook client). Before the fix it failed with exactly the ticket's('yes', 'unknown') == ('yes', 'tests_passed'). It passes after.test_older_servers_still_accept_the_test_run: the wire'stest_runkeeps exactly{command, exit_code}, andtest_run_atis an aware time.test_only_derived_shapes_are_accepted: a non-stringtest_run_atand a naivetest_run_at.make checkran four times in total (once in pre-commit, then three times in a row), all passing: 1571 tests.Learned
datetime.now) and a database time (now(),clock_timestamp()) can flip in tests. Keep each comparison on one clock.set(value) == {...}). To send a new field to them without breaking older servers, put it at the top level of the body, where older servers ignore unknown keys (doc_lookup,test_run_at).🤖 Generated with Claude Code
https://claude.ai/code/session_01Qc1CQTb8r67J32HpkhS43y