test(ui): run the live Playwright suite in its own CI job - #2938
Draft
toreysoloio wants to merge 2 commits into
Draft
toreysoloio wants to merge 2 commits into
toreysoloio wants to merge 2 commits into
Conversation
kagent-dev#2874 was reverted for making test-e2e too long. It put the browser lane inside that job, so its minutes landed in series on the job everything else waits on. This restores the work and moves the lane out. test-e2e-ui stands up its own cluster and runs beside test-e2e rather than after it, so a lane of about five minutes hides inside one of about ten instead of extending it. test-e2e is byte-identical to main again, ui.replicas=0 and five images included. The second cluster costs a runner, not wall-clock. The job is also cheaper than the one it left: three images where test-e2e builds five - controller, UI, and one runtime for the harnesses to point at - because the harness kinds that job needs differ in ways no browser journey can see. It brings its own fixture rather than borrowing the Go suite's. Three harnesses, all of one kind, so the template form still meets a cluster with more than one: that is the branch where it applies no labels unasked, and a single-harness fixture would have retired it silently. Verified by applying the fixture to a dev cluster and running the suite against the three-harness shape - 13 passed, including the admission journey. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Signed-off-by: Torey Scheer <[email protected]>
github-actions
Bot
added
testing
The first CI run measured the new job at 11m31s against test-e2e's 10m06s, which the README's "hides inside" claim does not survive. The cause is its own Blacksmith cache-key: the sticky disk came from a base snapshot with 1 cached layer, where test-e2e's came from a commit snapshot with 57, so it built three images from nothing - 332s for the step against 47s on the warm key. A separate key is still right, since the two jobs now run at the same time and would otherwise write one disk. What was wrong was a README that described the steady state as though it were the first run. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Signed-off-by: Torey Scheer <[email protected]>
github-actions
Bot
added
testing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🤖 written by Claude (start)
Re-lands #2874, which was reverted in #2910 because it added its minutes to
test-e2e. The suite and the product fixes are unchanged from #2874. The only difference is that the browser lane now runs as its owntest-e2e-uijob, in parallel withtest-e2e.Closes #2657
Closes #2828
Timings
The two jobs run in parallel. On the first run (
36041949897), both started at20:09:13, because neitherneedsthe other.test-e2etest-e2e-uiInstall Kagent, 332sThat first run is not the steady state.
test-e2e-uihas its own cache key so the two jobs don't write to one sticky disk at the same time, and a new key starts empty:Install Kagenttest-e2ekagent-e2etest-e2e-uikagent-e2e-uiFor reference,
test-e2eon the same runner type:maintodaymain, with the lane inside itrelease/v0.10.xThe lane cost about 51s when it ran inside
test-e2e, which is small next to that job's 5m27s–11m29s spread. Moving it out givestest-e2e, usually the longest job in a run, some headroom rather than a visible speed-up.Fixture
ui/playwright/live/fixtures/cluster.yaml.tmplbuilds three images (controller, UI and one runtime) instead of the fivetest-e2ebuilds. It declares three harnesses of one kind because a template applies a lone harness's labels unasked, which would stop the multi-harness path from being exercised.Testing
test-e2eandtest-e2e-uistart together and thattest-e2efalls in the 5m27s–11m29s range above.scripts/setup-cluster/setup-cluster.sh, then apply the CI fixture:cd ui && UI_LOOP_LIVE_URL=http://localhost:8080 yarn test:pw:live.kubectl get agenttemplate,harness,modelconfig -A | grep e2e-live. It should return nothing.🤖 written by Claude (end)