You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
CLI --load: opt-in error-rate / latency thresholds that set the exit code (for CI) #2745
A --load run that completes exits 0 whatever its error rate (documented in the CLI README). To use it as a CI gate today you have to parse the summary with --format json and jq and do the comparison yourself.
Proposal
Add an opt-in threshold so the exit code carries the verdict, for example:
--load-max-error-rate <0..1>: fail when errorRate is above the value (it is already the share of calls that were tool errors or thrown errors).
Possibly --load-max-p99 <ms> for a latency SLO. Decide whether that belongs in the same change.
When a threshold is breached, still print the full summary, then exit with a new dedicated exit code (or an existing one if one fits; check core/cli exit codes) and a structured error code naming which threshold failed. With no threshold set, behavior is unchanged.
Validation: the threshold flags require --load, and the value must be in range.
If #2744 lands first, add the flags there too, since both clients share core/cli/handlers/load-test.ts.
Priority score
Severity 2 (easy workaround with jq) + urgency 3 (wanted this milestone) + 1 (milestoned) = 6 → Medium.
Activity