Repository navigation
Add a runbook for one benchmark run on AWS - #3
Merged
Merged
Conversation
The routine from sign-in to teardown: confirm the account, configure and plan, wait for first boot, copy code up, run under tmux, watch the lifetime timer, copy results off, destroy and verify, and record the cost. It also records two behaviors the existing docs did not state. Changing max_lifetime_hours replaces the host, because the value is part of the first-boot script, and a plan against a running host can replace it when a newer Ubuntu image appears. The cost guardrails page said to raise the variable and apply again, which would have destroyed a running host's data, so it now points to the supported way to extend a running host. The runbook is deliberately generic and names no experiment. Co-Authored-By: Claude Sonnet 5.5 <[email protected]> Signed-off-by: Rob Grafrath <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes and why
This adds docs/run-a-benchmark.md, the routine for one benchmark run on an AWS sandbox host, from signing in to destroying the host and recording what it cost. It links the page from the README and corrects one sentence in the cost guardrails. The repository explains how to build the account and each stack, but not the routine for a single run, and the first real run will need one.
The page is deliberately generic and names no experiment.
Writing it turned up three behaviors that the existing docs did not state:
Does it make a host easier to leave running or easier to reach?
It documents one way to lengthen a running host's life (
sudo shutdown -c; sudo shutdown -h +N). The stated reason is that applying again would destroy the run, and the page requires scheduling a new timer in the same command line and noting the extension in the run log. It changes no stack, policy, or first-boot code, opens no port, and adds no permission.How I checked it
scripts/check-repo.py and
tofu fmt -check -recursivepass locally, and CI runs validate. I have not run the commands in the runbook against a real host, so the first real run should follow the page and correct it. The vCPU quota code (L-1216C47A) and the behavior ofshutdown --showare from the AWS and systemd documentation as I know it, so please confirm the quota code in the Service Quotas console.Left out on purpose
Checklist
tofu fmt -recursiveandtofu validatepass in each stack I changed, or I am relying on CI. No stack changed..tfvarsfiles, keys, tokens, account IDs, or personal addresses are committed.🤖 Generated with Claude Code