mthines · Oct 21, 2026 · Oct 21, 2026 · Oct 21, 2026 · Oct 21, 2026
@@ -56,7 +56,8 @@ it can never ship a flag that silently resolves to `undefined`.
 
 > **Every `example-*` key in this document is illustrative, not a real flag.**
 > Grep for one and you will find only this file. The registry holds exactly the
-> flags that gate live behaviour — today four rollout gates, all plain booleans:
+> flags that gate live behaviour — today four rollout gates, all plain booleans
+> (`insights-visual` also declares a 50/50 experiment that is not running):
 > `retention-policies`, `lore-explorer-instruments`,
 > `lore-explorer-duplicate-clusters` and `insights-visual`. It has also held entries since removed: the
 > rollout gate for the `/insights` route (deleted once that page became
@@ -154,7 +155,7 @@ stamps the **active span** with:
 | `feature_flag.provider.name`  | `lorekit-flags`       |                                                                     |
 | `feature_flag.context.id`     | `user_abc123`         | The `targetingKey`, when the evaluation context carries one         |
 | `feature_flag.result.variant` | `treatment`           | **This is the A/B dimension** — filter or group by it               |
-| `feature_flag.result.reason`  | `SPLIT`               | `SPLIT` = experiment bucketing ran; `STATIC` = no active experiment |
+| `feature_flag.result.reason`  | `split`               | `split` = experiment bucketing ran; `static` = no active experiment; `override` = a session override won. Lowercased from OpenFeature's `SPLIT`/`STATIC` by `flagReasonValue` |
 
 ...plus a `feature_flag.evaluation` span event carrying the same data (so a
 trace that evaluates several flags still shows each one on the timeline), and
@@ -209,6 +210,15 @@ for the rest of the session:
 | Attribute | Example | Notes |
 |-----------|---------|-------|
 | `feature_flag.<flagKey>` | `feature_flag.example-onboarding-flow = "treatment"` | One dynamically-named attribute PER FLAG, holding its variant |
+| `lorekit.feature_flag.reason.<flagKey>` | `lorekit.feature_flag.reason.example-onboarding-flow = "split"` | How that variant was decided — the same values as the span's `feature_flag.result.reason`. Filter on `split` to keep developer overrides out of an experiment |
+
+Each sync REPLACES the previous values (`removeSignalAttribute` then
+`addSignalAttribute`): the SDK's add appends, so a re-sync after an override
+flip used to leave both variants on every later signal.
+
+In PromQL a hyphen in the flag key becomes an underscore:
+`feature_flag.insights-visual` is the label `feature_flag_insights_visual`, and
+its reason is `lorekit_feature_flag_reason_insights_visual`.
 
 This is a genuinely different shape from the server-side span attributes
 above, on purpose: the OTel feature-flag semantic conventions
@@ -222,11 +232,21 @@ representation that doesn't collide, at the cost of not being spec-standard.
 Only the variant is attached, never an `object`-typed flag's whole payload —
 same "prefer `variant` over `value`" reasoning as the span hook.
 
-**This is what answers "did treatment convert better than control":** search
-or group Web Events by `feature_flag.example-onboarding-flow`, and compare
-whatever conversion event you're tracking (a `sign_up_completed` custom event,
-a specific page reached) between the `"control"` and `"treatment"`
-populations.
+**This is what answers "did treatment convert better than control":** group
+the conversion event you're tracking by `feature_flag.<flagKey>`, filtered to
+`lorekit.feature_flag.reason.<flagKey> = "split"`, and divide by each arm's
+exposed users from `feature_flag.exposed` — see
+[Running an experiment](#running-an-experiment).
+
+**Known gap — the initial page load.** The sync runs after React mounts, so
+signals the SDK records while a page first loads carry no flag tags: 192 of 219
+dashboard page views (88%), 115 of 151 navigation timings and all 31 browser
+errors over 2026-10-14..21. `@dash0/sdk-web` only attaches attributes to those
+when they are passed to `init()` as `additionalSignalAttributes`, and `init()`
+runs before the flags are known. Until that changes, do not compare page views,
+load-time web vitals or browser errors per variant; count the experiment's
+population with `feature_flag.exposed`, which is emitted after mount and always
+carries its variant.
 
 **Before this was wired (nothing appears):** if you evaluated flags server-
 side (Server Components) before `FeatureFlagsProvider` existed on this
@@ -235,6 +255,122 @@ provider mounts, no `feature_flag.*` attribute reaches RUM at all — only the
 server-side span hook fires. A `feature_flag.*` search on Web Events finding
 nothing is the correct, expected symptom of that gap, not a broken query.
 
+## Running an experiment
+
+Per-user splits need no storage: a signed-in user is bucketed by their Supabase
+user id (see [Targeting key resolution](#targeting-key-resolution)), so they see
+the same arm on every device for as long as the weights stand still.
+`insights-visual` is declared this way and is the worked example.
+
+### 1. Declare the arms, dark
+
+```ts
+experiment: {
+  enabled: false,
+  variants: [
+    { key: 'on', weight: 50, description: 'The new surface' },
+    { key: 'off', weight: 50, description: 'Control' },
+  ],
+},
+```
+
+`enabled: false` resolves every user to `defaultVariant` with reason `static`,
+so merging changes nothing. **List the treatment first**: the first arm owns
+buckets `[0, weight)`, so raising its weight only ever adds users to it
+(`bucketing.spec.ts` § "ramping an experiment"). Lowering it, or reordering the
+arms, moves users between arms mid-experiment.
+
+### 2. Count exposure in the resolver
+
+Call `useFeatureFlagExposure('<flag-key>')` in the copy-and-suffix resolver,
+never in one arm — the control arm's exposures are half the comparison. It emits
+`feature_flag.exposed` once per mount (and again if the variant changes), one
+task after the commit so it carries the real user id rather than the anonymous
+one:
+
+| Attribute | Example |
+|---|---|
+| `feature_flag.key` | `insights-visual` |
+| `feature_flag.result.variant` | `on` |
+| `feature_flag.result.reason` | `split` |
+
+### 3. Launch and ramp
+
+Set `enabled: true` and deploy. To ramp, raise the first arm's weight (and lower
+the second to keep the sum at 100). Visitors who are not signed in are bucketed
+by the `lorekit_flag_anon_id` cookie and can change arm when they sign in, so
+run experiments on authenticated surfaces.
+
+### 4. Measure
+
+Exposed users per arm — the population, and the sample-ratio check:
+
+```promql
+sum by (feature_flag_result_variant) (
+  clamp_max(
+    sum by (feature_flag_result_variant, user_id) (
+      increase({otel_metric_name="dash0.web.events", service_name="web",
+        event_name="feature_flag.exposed", feature_flag_key="insights-visual",
+        feature_flag_result_reason="split", user_id!="", user_id!~"anon:.*"}[$__range])
+    ),
+    1
+  )
+)
+```
+
+The clamped sum is the distinct-user shape that matches a SQL `uniqExact`
+on `dash0.web.events`; `count()` and `dash0.web.users` do not.
+
+**Sample-ratio mismatch is the regression signal.** With `n` exposed users and an
+arm weighted `p`, that arm's share should stay within `p ± 3·√(p(1−p)/n)` —
+±15 points at 100 users, ±7.5 at 400. Outside it, assignment or exposure is
+broken (a resolver that only one arm reaches, a cached page): stop reading
+outcomes until it is explained.
+
+Outcomes per exposed user, per arm, overrides excluded — here every Insights
+interaction. The exposure event names its arm `feature_flag_result_variant` and
+the outcome events `feature_flag_insights_visual`, so the denominator is
+relabelled before dividing:
+
+```promql
+sum by (feature_flag_insights_visual) (
+  increase({otel_metric_name="dash0.web.events", service_name="web",
+    event_name=~"^insights[.].*", lorekit_feature_flag_reason_insights_visual="split"}[$__range])
+)
+/ on (feature_flag_insights_visual)
+sum by (feature_flag_insights_visual) (
+  label_replace(
+    clamp_max(
+      sum by (feature_flag_result_variant, user_id) (
+        increase({otel_metric_name="dash0.web.events", service_name="web",
+          event_name="feature_flag.exposed", feature_flag_key="insights-visual",
+          feature_flag_result_reason="split", user_id!="", user_id!~"anon:.*"}[$__range])
+      ),
+      1
+    ),
+    "feature_flag_insights_visual", "$1", "feature_flag_result_variant", "(.*)"
+  )
+)
+```
+
+Server-side outcomes (errors, latency) group spans by
+`feature_flag.result.variant` instead.
+
+### 5. Finish
+
+Ship the winner as [Removing a flag](#removing-a-flag) describes: promote its
+arm into the resolver, delete the other, drop the registry entry — and the
+`useFeatureFlagExposure` call with it.
+
+### What this does not do
+
+- **No saved per-user assignment.** The arm is a pure function of flag key,
+  user id and weights. There is no way to pin one user to an arm across devices
+  (a session override pins one browser, and only for allowlisted developers in
+  production), and a non-monotonic weight change reassigns users.
+- **Weights change by deploy.** `registry.ts` stays the single source of truth;
+  there is no runtime percentage control.
+
 ## Cross-language flags (adding a language)
 
 Nothing above is TypeScript-specific in its _data_. `generated/flags.manifest.json`

@@ -1075,6 +1075,18 @@ Both the command and the surface are reported as **bounded ids**, never the
 command string: `CopyCommand` takes arbitrary text, which would become unbounded
 the moment a call site interpolates into it.
 
+### Feature flags (`feature_flag.*`)
+
+| Event | Emitted when | Attributes |
+|-------|--------------|------------|
+| `feature_flag.exposed` | A user reached a flag-gated surface, in either arm (once per mount, from the flag's resolver) | `feature_flag.key`, `feature_flag.result.variant`, `feature_flag.result.reason` (`static`/`split`/`override`) |
+
+The semantic-convention names rather than `lorekit.*`, because one event is one
+evaluation's outcome — what those names describe — and the server-side span hook
+uses the same three. Every RUM signal additionally carries
+`feature_flag.<flagKey>` (variant) and `lorekit.feature_flag.reason.<flagKey>`;
+see `docs/feature-flags.md` § Running an experiment.
+
 ---
 
 ## Browser auth events (`auth.*`)

@@ -57,20 +57,35 @@
     },
     {
       "key": "insights-visual",
-      "description": "Insights — the visual first version of `/insights`: ONE page-wide window, the Learn → Recall → Promote → Prune loop strip, the Pulse timeline (lessons written and records read on a shared time axis, chosen vs bulk), the recall flow (client → tool → scope type), and runs drawn as a weekday × hour rhythm plus swimlanes — with the existing 'earning its place' grid, runs list and operational health kept below. Gates the whole page body through the `InsightsView` resolver; with it off `/insights` renders exactly as before. Reads only routes that already ship (`/memories/activity`, `/read-activity`, `/usage`, `/usage/runs`, `/utility`), so there is no backend prerequisite and flipping it is a pure UI decision.",
+      "description": "Insights — the visual first version of `/insights`: ONE page-wide window, the Learn → Recall → Promote → Prune loop strip, the Pulse timeline (lessons written and records read on a shared time axis, chosen vs bulk), the recall flow (client → tool → scope type), and runs drawn as a weekday × hour rhythm plus swimlanes — with the existing 'earning its place' grid, runs list and operational health kept below. Gates the whole page body through the `InsightsView` resolver; with it off `/insights` renders exactly as before. Reads only routes that already ship (`/memories/activity`, `/read-activity`, `/usage`, `/usage/runs`, `/utility`), so there is no backend prerequisite and flipping it is a pure UI decision. Declared as a 50/50 experiment that is NOT running (`enabled: false` resolves every user to `off`): launching it is flipping that one boolean, and `on` is listed first so ramping its weight up never moves a user from `on` back to `off`. Exposure is measured by `feature_flag.exposed`, emitted by the `InsightsView` resolver — see docs/feature-flags.md § Running an experiment.",
       "type": "boolean",
       "variants": {
         "off": false,
         "on": true
       },
       "defaultVariant": "off",
-      "experiment": null,
+      "experiment": {
+        "enabled": false,
+        "variants": [
+          {
+            "key": "on",
+            "description": "The visual page",
+            "weight": 50
+          },
+          {
+            "key": "off",
+            "description": "The original page (control)",
+            "weight": 50
+          }
+        ]
+      },
       "owner": "@lorekit/web",
       "tags": [
         "dashboard",
         "web",
         "insights",
-        "rollout"
+        "rollout",
+        "experiment"
       ]
     }
   ]

@@ -83,3 +83,29 @@ describe('assignExperimentVariant', () => {
     expect(sawDifference).toBe(true);
   });
 });
+
+describe('ramping an experiment', () => {
+  it('never moves a user from the first arm to the second when the first arm only grows', () => {
+    // Cumulative bucketing: the first arm owns buckets [0, w). Raising w keeps
+    // every bucket it already owned, so a ramp from 10% to 50% only ADDS users
+    // to the treatment — the reason the registry lists `on` first.
+    const ramp = [10, 25, 50, 100].map(
+      (onWeight): Experiment => ({
+        enabled: true,
+        variants: [
+          { key: 'on', weight: onWeight },
+          { key: 'off', weight: 100 - onWeight },
+        ],
+      }),
+    );
+    for (let i = 0; i < 2_000; i++) {
+      const user = `user-${i}`;
+      let wasOn = false;
+      for (const stage of ramp) {
+        const isOn = assignExperimentVariant('insights-visual', stage, user) === 'on';
+        if (wasOn) expect(isOn).toBe(true);
+        wasOn = isOn;
+      }
+    }
+  });
+});
@@ -20,6 +20,8 @@
  */
 import { beforeEach, describe, expect, it } from 'vitest';
 import { evaluateFlag, evaluateFlagDetails, resetFeatureFlagClientForTests } from './client.ts';
+import { FLAG_REGISTRY } from './registry.ts';
+import type { FlagKey } from './generated/flags.generated.ts';
 
 describe('evaluateFlag', () => {
   beforeEach(() => {
@@ -62,4 +64,17 @@ describe('evaluateFlagDetails', () => {
     });
     expect(details).toMatchObject({ value: false, variant: 'off', reason: 'OVERRIDE' });
   });
+
+  it('keeps every declared-but-disabled experiment on its default for every user', async () => {
+    // A dark experiment must change nothing: no user lands on another arm and
+    // none is reported as SPLIT. Derived from the registry rather than naming a
+    // flag, so launching one is still a single `enabled: true` edit.
+    const dark = FLAG_REGISTRY.filter((flag) => flag.experiment && !flag.experiment.enabled);
+    for (const flag of dark) {
+      for (const targetingKey of ['user-a', 'user-b', 'user-c', 'user-d']) {
+        const details = await evaluateFlagDetails(flag.key as FlagKey, { targetingKey });
+        expect(details).toMatchObject({ variant: flag.defaultVariant, reason: 'STATIC' });
+      }
+    }
+  });
 });
@@ -50,6 +50,7 @@ export {
   ATTR_FEATURE_FLAG_RESULT_REASON,
   EVENT_FEATURE_FLAG_EVALUATION,
   METRIC_FEATURE_FLAG_EVALUATIONS,
+  flagReasonValue,
 } from './otel-attributes.ts';
 export type { FlagKey, FlagValue, FlagValueMap } from './generated/flags.generated.ts';
 export { FLAG_KEYS } from './generated/flags.generated.ts';
@@ -0,0 +1,16 @@
+import { describe, expect, it } from 'vitest';
+import { flagReasonValue } from './otel-attributes.ts';
+
+describe('flagReasonValue', () => {
+  it("lowercases OpenFeature's reasons to the semantic conventions' values", () => {
+    expect(flagReasonValue('STATIC')).toBe('static');
+    expect(flagReasonValue('SPLIT')).toBe('split');
+    expect(flagReasonValue('OVERRIDE')).toBe('override');
+  });
+
+  it('reports a missing reason as unknown, never an empty string', () => {
+    // An empty attribute value is indistinguishable from "not set" in a query.
+    expect(flagReasonValue(undefined)).toBe('unknown');
+    expect(flagReasonValue('')).toBe('unknown');
+  });
+});
@@ -37,3 +37,18 @@ export const EVENT_FEATURE_FLAG_EVALUATION = 'feature_flag.evaluation';
 
 /** `lorekit.feature_flag.evaluations` — counter dimensioned by key + variant. */
 export const METRIC_FEATURE_FLAG_EVALUATIONS = 'lorekit.feature_flag.evaluations';
+
+/**
+ * A resolution reason as a `feature_flag.result.reason` VALUE: OpenFeature's
+ * UPPERCASE reasons lowercased. "STATIC" and "SPLIT" become the semantic
+ * conventions' well-known `static` and `split`; this package's own "OVERRIDE"
+ * becomes `override`, a custom value the conventions permit but do not define.
+ *
+ * One function, because three surfaces carry the reason — server spans, the
+ * browser's per-flag RUM attribute, and the exposure event — and an experiment
+ * query filters all three with ONE vocabulary. An absent reason is `unknown`,
+ * never an empty string, so a query can still tell "not reported" from a value.
+ */
+export function flagReasonValue(reason: string | undefined): string {
+  return reason ? reason.toLowerCase() : 'unknown';
+}
@@ -28,6 +28,7 @@ import {
   ATTR_FEATURE_FLAG_RESULT_VARIANT,
   EVENT_FEATURE_FLAG_EVALUATION,
   METRIC_FEATURE_FLAG_EVALUATIONS,
+  flagReasonValue,
 } from './otel-attributes.ts';
 import { getFeatureFlagMeter } from './telemetry.ts';
 
@@ -66,13 +67,10 @@ export const featureFlagOtelHook: Hook = {
     const span = trace.getActiveSpan();
     const attrs = baseAttributes(hookContext);
     if (details.variant) attrs[ATTR_FEATURE_FLAG_RESULT_VARIANT] = details.variant;
-    // Lowercased: OpenFeature's own `StandardResolutionReasons` values are
-    // UPPERCASE ("STATIC", "SPLIT") by its own convention, but the OTel
-    // feature-flag semantic conventions' well-known `feature_flag.result.reason`
-    // values are lowercase ("static", "split") — see the spec link in
-    // `otel-attributes.ts`. Transform at this boundary only; `provider.ts` keeps
-    // the idiomatic OpenFeature casing for its own `ResolutionDetails.reason`.
-    if (details.reason) attrs[ATTR_FEATURE_FLAG_RESULT_REASON] = details.reason.toLowerCase();
+    // Lowercased by `flagReasonValue`: OpenFeature's reasons are UPPERCASE, the
+    // semantic conventions' values lowercase. Transform at this boundary only;
+    // `provider.ts` keeps the idiomatic OpenFeature casing.
+    if (details.reason) attrs[ATTR_FEATURE_FLAG_RESULT_REASON] = flagReasonValue(details.reason);
     if (span) stampSpan(span, attrs);
     recordEvaluationMetric(hookContext.flagKey, details.variant);
   },