docs: the container never slept - the ~8.5min tail was a backfill artifact (#120) - #155
Merged
Merged
Conversation
…ifact (#120) #120 published an "~8.5 minute" idle tail from `containersUsageAdaptiveGroups`, read ~14 minutes after the test render that produced it. Re-read now that the dataset has settled, the same window tells a different story: the placement that render woke (`a20877a2`, 2026-08-03T22:35Z) was still allocated 29.3 hours later - 352 of 352 five-minute buckets present, zero gaps, 98.1% of a 1 GiB instance's full allocation - against a total of 5 worker requests in that entire window. There is no tail. The container simply never sleeps, which is exactly what the SIGTERM diagnosis in #154 predicts: `@cloudflare/containers` sends SIGTERM and never escalates, node is PID 1 and ignored it, so the instance only ever goes away when a deploy replaces the placement. Two claims in CLAUDE.md were wrong as a result, and both were cost claims: - "That cost ~$28/month for weeks" implied the drain fix ended it. Billing says the instance ran at 100% of a 4 GiB day on EVERY full day from 2026-07-20 through 2026-08-03 (95.3 - 99.0 GiB-hours/day against the 96.0 a 4 GiB instance bills for 24h), including the five days after that fix deployed on 2026-07-29. The bill was still being paid; the 2026-08-03 downsize to `basic` cut it ~4x rather than ending it. The drain guard stays - the hazard is real - but it was never the load-bearing cause. - The disk-to-memory ratios "2.0" and "4.0" are dashboard units. Queried in bytes, which is what the GraphQL dataset returns, they read 1.86 and 3.73. Adds the SIGTERM cause and the backfill trap as their own bullets, so the next person reading absence-of-bucket as sleep has the rule in front of them. Measured via the `containersUsageAdaptiveGroups` and `workersInvocationsAdaptive` GraphQL datasets on account 3e467e5d..., 2026-08-05. NOT a behaviour change: the fix itself is 343df53, already on main and NOT yet deployed. Production still runs worker version 2e67470f and container app version 3 (image sha256:16205eda..., updated 2026-08-03T21:24:01Z), both of which predate the handler. Verified: pnpm run verify green, exit 0 - 1588 app tests passed / 3 skipped, 13 worker, 4 container (including "exits promptly on SIGTERM"), 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01QZvNS2H4cbaZk46ybA7hbj
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The number in #120 was an artifact of reading the dataset too early
#120 published an ~8.5 minute idle tail, measured from
containersUsageAdaptiveGroupsabout 14 minutes after the render that produced it.22:50Zwas recorded as absent - asleep.Re-read now that the dataset has settled, that bucket is there, and so is every bucket after it. The placement that render woke -
a20877a2, first bucket2026-08-03T22:35:00Z- was still allocated 29.3 hours later:a20877a2)Five requests cannot hold a container open for 29 hours against
sleepAfter = "20s". There is no tail. The container never sleeps at all - which is precisely what the SIGTERM diagnosis in #154 predicts:@cloudflare/containerssends SIGTERM and never escalates to SIGKILL, node is PID 1 and ignored it, so the instance only ever goes away when a deploy replaces the placement.Two cost claims in CLAUDE.md were wrong
1. The drain fix did not end the bill. Daily allocated memory,
applicationId = a039bbb1, against the 96.0 GiB-hours a 4 GiB instance bills for a full day:100% of every full day, including the five days after the drain fix deployed on 2026-07-29. The ~$28/month was still being paid right up to the 2026-08-03 downsize, which cut it ~4x rather than ending it. The drain guard stays - the hazard is real and
worker.spec.tsshould keep testing it - but it was never the load-bearing cause of the bill.2. The disk-to-memory ratios are dashboard units. Queried in bytes, which is what the GraphQL dataset actually returns,
standard-1reads 1.86 andbasicreads 3.73, not 2.0 and 4.0. Both forms are now stated so neither reading surprises anyone.What this PR does
Docs only - no behaviour change. It corrects those two claims and adds two bullets that were missing: the SIGTERM/PID-1 cause, and the backfill trap ("wait at least an hour before reading absence as sleep, and confirm with
placementIdcontinuity rather than bucket presence alone").Status of the fix itself
Still not deployed. 343df53 is on
main; production runs worker version2e67470fand container app version 3 (imagesha256:16205eda...,updated_at 2026-08-03T21:24:01Z), both of which predate the handler. #120 stays open until apreview:deployand a settled re-read of the buckets.Verification
pnpm run verify- exit 0. 1588 app tests passed / 3 skipped (196 files), 13 worker tests, 4 container tests includingexits promptly on SIGTERM, 0 warnings.🤖 Generated with Claude Code
https://claude.ai/code/session_01QZvNS2H4cbaZk46ybA7hbj