Repository navigation
CI: shard the UI tests across machines and merge the blob reports - #11
Merged
Merged
Conversation
Each browser's UI tests now run on 2 machines (--shard=1/2, 2/2), the API tests stay in one job: 7 shard jobs instead of 4 project jobs. A Tests (<project>) job per project merges that project's blob reports into the same JUnit XML and JSON report an unsharded run wrote, so the ruleset's required checks and the artifacts Quality Intelligence imports keep their names. src/ci/check-merged-results.mts fails that job when a shard is missing or a test failed. The final job still merges all blob reports into one HTML report. Checked locally: chromium as 2 shards merged -> the same 24 test keys as the unsharded main run 41627424324, retry attempts kept (TC-116 flaky). Co-Authored-By: Claude Opus 5.5 <[email protected]>
The logged-in test user's profile address is already in the billing form, so Proceed was enabled at once. When GET /postcode-lookup answered late (CI), the order went out with the profile address (Stephansplatz 1, Vienna) and POST /invoices answered 422 "The city does not belong to the selected country". enterBillingAddress now waits for the lookup of the complete address and for its street to be in the form. Reproduced locally by delaying the lookup 3 s: old steps -> the same 422 with the profile address in the body; fixed page object -> order confirmed. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Owner
Author
|
CI failure: TC-116 (chromium) was a real test bug, not a flake. The billing form starts out holding the logged-in user's profile address, so Proceed is enabled right away. When |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds Playwright sharding to CI.
What changes
shardsjob (7 jobs): API tests in one job, each browser's UI tests split with--shard=1/2and--shard=2/2. Each uploads a blob report.testsjob, one per project (Tests (api),Tests (chromium),Tests (firefox),Tests (webkit)): merges that project's blob reports withplaywright merge-reports --reporter junit,jsonand runssrc/ci/check-merged-results.mts, which fails the job if a shard is missing or a test failed.reportjob: still merges all blob reports into one HTML report.Why it is built this way
Tests (<project>). Those check names stay the same, so the ruleset doesn't need to change.junit-<project>andplaywright-json-<project>artifacts that Quality Intelligence imports keep their names and content.Checked locally
npm run checkgreen.The trade-off: 7 jobs × 2 workers instead of 4 × 2 against the public demo shop. The README explains how to change the shard count.
🤖 Generated with Claude Code