Skip to content

ci: get slow integration test results back sooner - #1446

Merged
MicBun merged 1 commit into
mainfrom
slow-tests-in-parallel
Oct 9, 2026
Merged

MicBun merged 1 commit into
mainfrom
slow-tests-in-parallel

Conversation

@MicBun

@MicBun MicBun commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Description

The slow integration tests ran as one job: every slow package in turn, about 135 minutes on a typical run. They now run as three jobs side by side:

Job Packages Typical time today
order_book tests/streams/order_book 51 min
streams tests/streams 35 min
rest every other slow package 45 min

rest is listed at run time with go list, minus the two packages that have their own job, so a new test package lands there without editing the workflow. Together the three jobs cover the same 24 packages as before. A retry now reruns only its own job's packages, where a failure used to restart all of them.

When a job's tests fail, the Docker diagnostics now also print the last lines of the test database container, kwil-testing-postgres. A run that stalled on that container showed only docker ps, which said it was up and nothing more.

The repository has no required status checks, so the new job names (slow-integration-tests (order_book) and so on) need no settings change.

Related Problem

Part of:

How Has This Been Tested?

  • actionlint reports nothing for the changed step.
  • Running the shard selection in bash for each shard lists the expected packages, and an unknown shard fails.
  • The slow jobs do not run on a pull request that touches only the workflow, and the run-slow-tests label the workflow mentions does not exist in this repository. I started the workflow by hand on this branch instead, which runs all three jobs. All three passed: order_book in 41 minutes, streams in 43 and rest in 50, so the whole run took 50 minutes:

Summary by CodeRabbit

  • Chores
    • Split slow integration tests into three parallel groups, with each group running its selected packages.
    • Failed test runs now include recent PostgreSQL container logs to help diagnose issues.

@MicBun MicBun self-assigned this Oct 9, 2026
@holdex

holdex Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

Time Submission Status

Member # Time Running Total Status Last Update
MicBun 1h ✅ Submitted Oct 9, 2026, 9:04 AM

Submit or update total time with:

@holdex pr submit-time 2h

Add time on top of previous submission with:

@holdex pr add-time 1h30m

See available commands to help comply with our Guidelines.

@coderabbitai

coderabbitai Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 2bd8b5ca-10d0-40a1-8686-b63181e511e3

📥 Commits

Reviewing files that changed from the base of the PR and between e848d39 and 58f7fa2.


📒 Files selected for processing (1)
  • .github/workflows/ci.yaml

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.



📝 Walkthrough

Walkthrough

The CI workflow now runs slow integration tests in three shards. Each shard selects its package list and runs it through the retry loop. If tests fail, the workflow prints the last 100 lines of the PostgreSQL container logs.

Changes

Slow Integration Test Sharding

Layer / File(s) Summary
Shard selection and execution
.github/workflows/ci.yaml
The workflow defines order_book, streams, and rest shards. Each shard selects its package list; unknown shard names and empty lists fail. The retry loop tests the selected packages and prints PostgreSQL logs on failure.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Other


Merge Risk: ⚪ Minimal · up to 58f7f

The slow-test shards retain the previous package coverage. No actionable merge risk remains after normal checks.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly describes the main change: parallelizing slow integration tests to return results sooner.
Docstring Coverage Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.


✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Commit to this branch
  • Create a new PR

🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR


  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@MicBun

MicBun commented Oct 9, 2026

Copy link
Copy Markdown
Contributor Author

@holdex pr submit-time 1h

@MicBun
MicBun merged commit c14b49b into main Oct 9, 2026
14 checks passed
@MicBun
MicBun deleted the slow-tests-in-parallel branch October 9, 2026 09:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Problem: developers can't get node PR checks back within an hour

1 participant