Repository navigation
feat(flags): launch the insights-visual experiment at 50/50 - #723
Merged
Merged
Conversation
Switch insights-visual's experiment on: each signed-in user is assigned the visual or the original Insights page by a hash of their user id, and a developer override still wins. Regenerates the flag manifest, updates the flag description and docs, and adds a registry-derived test that every running experiment reports SPLIT and stays near its declared weights. Co-authored-by: dash0-dev[bot] <257284812+dash0-dev[bot]@users.noreply.github.com>
Contributor
Author
✅ No issues foundZero-delta re-check at e7471ad: the edited PR body now lists the provider.spec.ts rollback test, clearing the only open gate. No open threads, CI green, code unchanged since the last review. Progress: open review threads 1 → 0 → 0 across the last 3 reviews Review details
Nothing to report — standards (skipped), measurability (skipped), severity, 0 files skipped.
|
![dash0-dev[bot]](https://avatars.githubusercontent.com/in/2731631?s=60&v=4){kind=small}
…der fixture test Addresses dash0-dev's comment: #723 (comment) Refs: #723 Co-authored-by: dash0-dev[bot] <257284812+dash0-dev[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The visual Insights page (#714) and its measurement (#716) are live, but the experiment was declared with
enabled: false, so every user still gets the original page. This starts the 50/50 split.What changed
insights-visual:experiment.enabled: true— each signed-in user getsonoroffby a hash of their user id (same arm on every device); a developer override still wins. Weights stay 50/50 withonlisted first, so a later ramp only adds users toon.flags.manifest.json; updated the flag description anddocs/feature-flags.md("runs", not "not running").client.spec.ts: every running experiment reportsSPLITand stays within ±5 points of its weights over 2,000 users.provider.spec.ts: a disabled experiment (enabled: false) sends every user back to the default variant with reasonSTATIC— the rollback path, pinned independently of the live registry now that no registry flag is dark.How to verify
cd packages/feature-flags && npx vitest runandnode --experimental-transform-types scripts/codegen/gen-feature-flags.mjs --checkNotes for reviewers
Both arms render the
Insightsh1 the e2e specs assert, so a test user in either arm passes. User-facing docs andllms.txtare unchanged on purpose: the pages that mention Insights describe the cost line and the delivered × chosen grid, which both arms show. To stop the experiment, setenabledback tofalse.