diff --git a/.github/workflows/canary.yml b/.github/workflows/canary.yml index bcad7db..46968ae 100644 --- a/.github/workflows/canary.yml +++ b/.github/workflows/canary.yml @@ -2,7 +2,7 @@ name: canary on: schedule: - - cron: "41 6 * * *" # once daily; avoid :00 so it doesn't pile up with everyone else's + - cron: "41 6 * * 1" # weekly, Mondays (every run bills a Scraping Browser session); avoid :00 so it doesn't pile up with everyone else's workflow_dispatch: {} # dispatch manually at least once before trusting the badge jobs: diff --git a/CHANGELOG.md b/CHANGELOG.md index bc46de5..6191565 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,12 @@ rather than being a silent violation of that. ## [Unreleased] +### Changed + +- CI only: the canary bills a Scraping Browser session (`PERPLEXITY_CDP_ENDPOINT`), + so it now runs weekly (Mondays 06:41 UTC) instead of daily; a manual dispatch + still runs it. + ## [0.2.0] — 2026-10-16 > **Behaviour changes.** `--retries`, `--retry-delay`, diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index da66d80..9a3e53b 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -4,6 +4,6 @@ 2. Run `python3 smoke_test.py` (or `pytest tests/test_smoke.py`). It is offline and must pass with **no** engine installed; install one engine (`pip install -r requirements-playwright.txt`, or `-selenium` / `-puppeteer`) to exercise its code too. 3. **If you touched `page_parser.py`**: every parser check runs on real captures in `tests/fixtures/` (the `/rest/article/` API for a classic Page and a Discover article, the Discover feed, and a Cloudflare challenge page). If the site changed shape, save a scrubbed capture of what it serves now next to them, update the parser to match, and add a `smoke_test.py` check against the new fixture. Don't change the parser on a guess without a capture behind it. 4. Keep the three engines behaving identically: same exit codes, same `Product` schema, same CLI flags. The fetch logic itself lives once in `page_flow.fetch_article()`; an engine only supplies its driver's own page operations. If one engine has to diverge (see `selenium_scraper.py`'s CDP-credential limitation), say why in a comment. -5. Open a PR. CI runs the offline suite on two Python versions, builds the wheel and the Docker image, and runs one `engine-smoke` job per engine in its own virtualenv. A daily `canary` job scrapes the live site over the Scraping Browser API when the repo has the `PERPLEXITY_CDP_ENDPOINT` secret (see `TESTING.md`). +5. Open a PR. CI runs the offline suite on two Python versions, builds the wheel and the Docker image, and runs one `engine-smoke` job per engine in its own virtualenv. A weekly (Mondays) `canary` job scrapes the live site over the Scraping Browser API when the repo has the `PERPLEXITY_CDP_ENDPOINT` secret (see `TESTING.md`). Bug reports and feature requests: open an issue. Please include the exact command you ran (without credentials) and the `.meta.json` sidecar from the run — or say that none was written, which is itself informative (see `output_writer.finish_run`). If an article came back blocked or empty, a `--dump-html` capture is the most useful thing you can attach.