[jehrr]

Fixes from the 2026-10-14 audit of main at 1eb8327 (unchanged since the audit).

Problems and resulting behaviour

Finding	Before	After
H1 canary	14 scheduled runs (2026-10-01..14, and again 10-15) all exited 3 on SHEIN's /risk/action/limit and went green	canary-datacentre (refusal expected): exit 3 is a notice; 1/2/4/5 or < 10 rows are red. New daily canary-live: needs the SHEIN_CDP_ENDPOINT secret and fails with ::error:: when it is missing; 3/4/5 or thin rows are red. Assertions moved to .github/canary_assert.py so they can be run and tested offline
H2 parser failure	gbRawData stating sum=500 with no parseable products: exit 4, status: empty	exit 5, stop_reason: parser_found_nothing; a listing stating 0 is still exit 4. Scraper API path now records the stated total too
M1 atomic writes	an injected failure mid-json.dump left [{"partial	temp file in the target dir, fsync, os.replace, cleanup; json, csv and .meta.json. New file 0o666 & ~umask, existing file keeps its mode
M2 CSV formulas	=HYPERLINK(...), +cmd, @SUM(A1) written verbatim	apostrophe prefix in CSV only, numbers untouched, JSON untouched, csv_cells_escaped in the sidecar under the caller's extra
M3 mirror validators	selenium/pyppeteer accepted --retries -1 (0 navigations, exit 4) and negative delays/budgets	the three engines share page_flow.positive_int/nonnegative_int/nonnegative_float; exit 2 naming the flag. --proxy-block-retries validated in all three
M4 SHEIN_URL	.env URL silently replaced a typed --query dress	ignored with a warning when --query/--category is typed; still fills --url when nothing is typed (page_flow.prefer_typed_target, called after env_config.apply_env in every engine)
M5 private citations	~116 lines in 20 files	removed or restated in plain words in every tracked file, including released CHANGELOG sections (only the citations were touched). New guard assembles the banned tokens from pieces. Commit history not rewritten. v0.1.0 release body had none
M6 "credentials last about a day"	README, a client error message, and the reason canary-cdp was dispatch-only	removed; the 401 message says "may have expired or been revoked"; the live canary is scheduled
L1	--allow-empty rate-limited run recorded stop_reason: blocked	rate_limited; README wording matches
L2	diff_runs JSON vs CSV of identical rows: changed: 3	CSV rows normalised (numbers, empty cells, formula escapes): changed: 0
L3	Playwright --fp-tags help said 'Windows,Chrome'	ONE OS-family tag, e.g. 'Windows'
L5	tests.yml ran a narrower inline grep	runs .github/ci_checks.py; the scan refuses to pass after reading 0 files
L6	captures/a.html, root dump.html not ignored	ignored by kind (captures/ dumps/ out/ runs/ tmp/ ..., /*.html except landing.html); pinned both ways
L7	README silent on missing --pages/--concurrency/--mode	one paragraph explaining why

Repo description shortened from 29 to 25 words (gh repo edit).

Evidence

• Offline suite: 108/108 in a core-only venv; 108/108 in a venv per engine (playwright, selenium, pyppeteer installed from their requirements, no browser download); 108/108 in a copy shaped like the Docker build context (no .git, no .github).
• python3 <engine>.py --query x --retries -1 -> error: argument --retries: must be >= 0 in all three engines.
• ci_checks.py: credential scan passed (43 files); --sample-check OK.
• Controls (scratch copy, PYTHONDONTWRITEBYTECODE=1, no __pycache__): each fault turned the suite red on the named check, 13 of 13 — parser-failure detection off; rate_limited reason reverted; write_json back to open(..., "w"); flat 0o644 mode; CSV escape off; housekeeping merged over caller's extra; selenium --retries type=int; typed-target helper disabled; a citation re-added; canary judge excusing exit 4; diff_runs reading raw CSV; captures/ removed from .gitignore; tests.yml back to a grep.
• No paid calls, no proxies, no credentials, no request to shein.com or 2Captcha.

Not changed

• Per-run (not per-page) paid-solve budget (--max-solves): documented design, all call sites checked before purchase (audit L10, owner decision).
• --cdp-endpoint + --proxy and --scraper-api + browser flags still warn and drop rather than refuse (L9, low).
• pyppeteer setUserAgent / client hints (L4) and the Fingerprint API's current tag rules (L3 beyond the help text): need live/keyed checks.
• Donor-repo provenance names in comments (L8, comments only).
• smoke_test.py remains allowlisted wholesale in the credential scan (L5 remainder).
• No version bump, tag or release; schema/sinks (T11).
• Open question from the audit: whether SHEIN states a nonzero results.sum on a genuinely empty search. The new rule assumes it does not; no empty-search capture exists to check.

Needs the owner

canary-live will be red until a SHEIN_CDP_ENDPOINT repository secret is added (optionally TWOCAPTCHA_KEY for the /risk/challenge solve). That is intended: it is the only job that proves SHEIN still serves rows.

🤖 Generated with Claude Code