Repository navigation
Full live QA suite against the test account - #38
Conversation
Dispatch with suite=full to run scripts/live_qa.py against the test account: every read command in each format, pipelines, poll, refusal paths, checkpoints and checkpoint_ack, a second profile, and the CLI post lifecycle with a final sweep of test posts. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6fe8e5e70b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| def cli_reads() -> tuple[str, str]: | ||
| me = data(cli("whoami")).get("username", "") | ||
| cli("profiles") |
There was a problem hiding this comment.
mb blogs in the full suite
When suite=full, the workflow runs live_qa.py instead of live_smoke.py, but this read sequence never invokes mb blogs; that command remains only in live_smoke.main(), which is not called by the full script. Consequently, the supposedly comprehensive pre-release run can pass even if the blogs command is broken, so add it to cli_reads() or otherwise run the quick-smoke checks as part of the full path.
AGENTS.md reference: AGENTS.md:L368-L368
Useful? React with 👍 / 👎.
| lines = [line for line in out.splitlines() if line.strip()] | ||
| parsed = all(_is_json(line) for line in lines) | ||
| step( | ||
| "poll emits JSON lines and stops on ctrl-c", | ||
| parsed and "Traceback" not in err, |
There was a problem hiding this comment.
If poll exits or crashes cleanly before writing stdout, lines is empty and all(...) evaluates to true; the check also ignores the child return code. The full QA suite therefore reports that polling emitted JSON and stopped correctly even when it produced no events at all. Require a nonempty line list and a successful post-interrupt exit before passing this step.
AGENTS.md reference: AGENTS.md:L368-L368
Useful? React with 👍 / 👎.
1 participant
Adds
scripts/live_qa.py. Run it withgh workflow run live-smoke.yml -f suite=full; the weekly schedule still runs the quick smoke.Coverage
whoami,profiles,guide;--before,--sinceandcheck; mentions, photos, discover and its collections;user show, following, discover, is-following, muting, blocking;timeline | lookup posts -; usernames with comments piped tolookup users -;pollstopping on Ctrl-C.mb authwithout a terminal.checkpoint_ackapplying and then being idempotent; the read-only server refusingpost_createandcheckpoint_ack; verbose reads.mb --profile qa auth -from stdin; both profiles listed.mb live smokepost, and the sweep now fails when it can't list the blog's posts instead of passing silently.It does not run social writes (follow, mute, block, reply) or media uploads.
The
suiteinput reaches the shell through an env var, not template interpolation. Timeout raised to 30 minutes.🤖 Generated with Claude Code