Repository navigation
ci: run the test suite as parallel sections by area - #1101
Merged
Merged
Conversation
Split the single Run Tests job into a matrix of sections (core-data, core-extensions, core-theme, core-ui, features-workspace, features-nosql, features-app, rest) defined in scripts/ci/test_sections.sh. 'Run Tests' stays as the single required check and passes only when every section passes; a coverage job fails if a test file is in no section or in two. Tests in a directory no section lists still run in 'rest'.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The CI test job ran all 297 test files one after another on a single runner, so one slow or hung test held everything up. Tests now run as parallel jobs by area.
scripts/ci/test_sections.shdefines the sections as sets of test directories:core-data: database, storage, security, actions, app, json, csv, utilcore-extensions: extensions, market, sdui, updater, features/extensions, features/updatercore-theme: theme, shared widgetscore-ui: motion, layout, ui, editor, widgets, platform, demofeatures-workspace: workspace, sql_workspaces, sqlite, mysql, postgresqlfeatures-nosql: mongodb, redis, nosqlfeatures-app: main_screen, connections, settings, command_palette, onboarding, macos, helprest: everything no section claims (so a test in a new directory still runs)ci.yml: atest-sectionsmatrix runsflutter test --exclude-tags=golden <section paths>per section (fail-fast: false, so every section reports).build-linux-releasestill depends on it.Test sections cover all testsrunsscripts/ci/test_sections.sh check, which fails if a test file belongs to no section or to two.CONTRIBUTING.mdexplains how to run one section locally.Notes
pub get(about a minute), so total runner time grows while wall-clock time drops to roughly the slowest section.