feat(hogql): opt flagged users into the native events cluster by vdekrijger · Pull Request #119468 · PostHog/posthog

vdekrijger · 2026-10-19T10:49:16Z

Problem

The native events cluster can only be tried by sending each query to the API with explicit modifiers, so PostHog staff cannot click through the app on it.
The ClickHouse owner and the engineers driving the migration want to use the app on the new cluster, starting with themselves, and widen the group as checks pass.
Builds on #116838 (typed queries) and #115778 (async lifecycle), which made app queries able to run on the native cluster.

Changes

A user in the native-events-cluster-queries feature flag now runs their app queries on the native events cluster and the native JSON events table.
The opt-in applies only when all of these hold:
- the project is in CLICKHOUSE_NATIVE_EVENTS_TEAMS and the native host is set;
- the query type is one the native cluster supports;
- neither the query nor a nested source sets useNewEventsCluster or useNewEventsSchema;
- neither the query nor a nested source is raw HogQL;
- the request does not come from a personal API key (OAuth clients such as MCP are included);
- the project does not set its own useNewEventsSchema default.
Raw HogQL stays on its current route, because its workload depends on the tables it reads (logs and metrics are not analytics) and is only known after printing.
The opt-in keeps the query's other modifiers, such as personsOnEventsMode.
Every other query keeps its current route, so the flag never turns a working query into an error.
The flag does nothing until CLICKHOUSE_NATIVE_EVENTS_ASYNC is on. Dashboard tiles fill through an async run and read back from the cache, so both steps must agree on one route; with the gate off they would not.
Native results, flagged or explicit, skip the cache-warming freshness index, which tracks only legacy entries, so a native refresh never delays warming for everyone else.
The query API and the insights API mark their requests interactive, and process_query_model resolves the opt-in from the user they already pass, before the runner computes the cache key. Native and legacy results therefore keep separate cache entries, and a saved insight's query is parsed once.
Setting the modifier inside the runner's per-user defaults was rejected: the runner applies them after the cache lookup, so native results would land in the legacy cache entry.
Saved insights and dashboard tiles loaded through the insights API take the same opt-in, so once the async gate is on a flagged user sees native numbers on dashboards too.
The opt-in unwraps InsightVizNode and DataTableNode, which saved insights store, and checks the query that runs. Runnable wrappers such as FunnelsActorsQuery are checked as themselves, so the flag never opts in a query its runner would refuse.
Shared dashboards, exports, subscriptions, alerts, cache warming and server-side PostHog AI execution keep the legacy route. PostHog AI tiles a flagged user's browser renders through /query take the flag like any other request.
The flag reads the request's access method itself, so personal API key traffic stays legacy on both paths, and it evaluates through posthog_feature_flag_enabled like other server-side gates.

Warning

To roll back, turn the flag off first. Only remove the project from CLICKHOUSE_NATIVE_EVENTS_TEAMS or clear the native host after queued queries drain. Workers recheck eligibility and fail a queued native query rather than silently moving it to another cluster.

Note

The flag does not exist yet. Create native-events-cluster-queries in PostHog, targeted by email, before this has any effect.

How did you test this code?

Test rationale:

A parameterized API test in TestNewEventsClusterPilot runs one query with the flag off and on. It catches the opt-in not reaching the runner, and native results sharing the legacy cache key. It fails when the query API stops marking its requests interactive.
Unit tests check that the flag waits for the async gate, and that it targets the signed-in user's distinct ID and email with organization and project groups, evaluated locally.
A parameterized unit case in TestNewEventsClusterRouting catches the flag dropping a query's other modifiers, or forcing raw HogQL, unsupported query types, explicit schema choices or nested opt-outs onto the native route.
A parameterized API test loads a saved insight through the insights API with the flag off, on, and on with a personal API key. It fails when the insights API stops marking its requests interactive, when the opt-in stops looking through the wrapper node, when API key traffic takes the flag, or when a native run updates the freshness index.
A unit case covers a FunnelsActorsQuery that the flag must leave alone.
Not run: the ClickHouse-backed suites, because the local ClickHouse replicas were read-only.

Release status

This change is behind a feature flag and is not available to users

Docs update

None. No doc covers the native cluster opt-in yet.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Agent: Claude Code, Opus 5.5 (claude-opus-5-5)

Skills: /writing-tests, /writing-pr-descriptions.
A duplicate search for open native-cluster flag PRs found none.
The first idea, setting the modifier in create_default_modifiers_for_user, was dropped after reading the cache path (see Changes).

🤖 Generated with Claude Code

…uster A user in the native-events-cluster-queries flag gets useNewEventsCluster and useNewEventsSchema on their query API requests, for eligible projects and supported query types, unless the query sets either modifier itself. The opt-in reaches the runner as constructor modifiers, before the cache key is computed, so native results keep their own cache entries.

posthog

PostHog Review · Standard · reviewed 40bc8e0: 3 Should fix.

This is the only automatic review while this pull request is a draft. Comment @posthog review to review new changes sooner. Otherwise, a review of the new changes runs when you mark it ready for review.

github-actions · 2026-10-19T10:53:15Z

🤖 CI report

⚠️

Trunk lane — backend Python lane

This PR is assigned to the backend Python lane. It runs backend Python tests and may merge in parallel with PRs in other lanes.

⚠️

Duplication (Python) — 33 new duplicated blocks (worst 187 tokens)

New Python code duplication introduced by this branch. Fails at 70+ tokens in app code, or 150+ tokens when both copies live in test files. Advisory while the gate proves itself: extract a shared helper instead of copying.

⚠️

Duplication (TypeScript) — 10 new duplicated blocks (worst 1387 tokens)

New TypeScript code duplication introduced by this branch. Fails at 70+ tokens in app code, or 150+ tokens when both copies live in test files. Advisory while the gate proves itself: extract a shared helper instead of copying.

First copy	Second copy	Lines	Tokens
`frontend/src/products.tsx:394`	`products/ai_observability/manifest.tsx:219`	121	1387
`frontend/src/products.tsx:429`	`products/ai_observability/manifest.tsx:257`	87	1029
`frontend/src/products.tsx:1332`	`products/ai_observability/manifest.tsx:355`	83	630
`frontend/src/products.tsx:75`	`products/ai_observability/manifest.tsx:180`	38	237
`products/signals/frontend/inbox/signalSourcesLogic.ts:688`	`products/signals/frontend/inbox/signalSourcesLogic.ts:753`	64	140
`frontend/src/products.tsx:2700`	`products/ai_observability/manifest.tsx:486`	21	132
`products/signals/frontend/inbox/signalSourcesLogic.ts:614`	`products/signals/frontend/inbox/signalSourcesLogic.ts:688`	63	130
`frontend/src/scenes/hog-functions/email-templater/EmailTemplater.tsx:240`	`frontend/src/scenes/hog-functions/email-templater/EmailTemplater.tsx:795`	24	84
`products/ai_observability/frontend/datasets/AIObservabilityDatasetsScene.tsx:76`	`products/ai_observability/frontend/datasets/DatasetItems.tsx:134`	15	75
`products/signals/frontend/inbox/signalSourcesLogic.ts:644`	`products/signals/frontend/inbox/signalSourcesLogic.ts:783`	35	74

✅ MCP agent API — no agent API changes

This PR no longer changes the tools, params, scopes or annotations agents see.

⚠️

MCP snapshots — 4 updated (4 modified, 0 added, 0 deleted)

Snapshots: MCP unit test snapshots updated

Changes: 4 snapshots (4 modified, 0 added, 0 deleted)

What this means:

Snapshots have been automatically updated to match current output

Next steps:

Review the changes to ensure they're intentional
If unexpected, investigate what caused the output to change

Review snapshot changes →

The rollout opt-in now starts from the query's effective modifiers, including nested sources, so settings such as personsOnEventsMode survive and a nested explicit opt-out keeps its route. HogQL queries are no longer opted in automatically, because their workload depends on the tables they read and is only known once they are printed.

trunk-io · 2026-10-19T11:07:41Z

_{View Full Report ↗︎ ⋅ Docs}

…cle' into vasco/native-events-user-flag

…events-user-flag

…nto vasco/native-events-user-flag

…events-user-flag

posthog

PostHog Review · Standard · reviewed 388d983: no findings.

This Standard review started from the posthog-review label. Comment @posthog review --deep for a Deep review, or @posthog help to see all commands.

…events-user-flag

posthog

PostHog Review · Standard · reviewed 57c25da: no findings.

This Standard review started from the posthog-review label. Comment @posthog review --deep for a Deep review, or @posthog help to see all commands.

…lout flag

…events-user-flag

…events-user-flag # Conflicts: # posthog/api/services/query.py

…events-user-flag

…nd dashboard tiles The insights API now passes the user's native cluster opt-in into calculate_for_query_based_insight, and the opt-in looks through wrapper nodes such as InsightVizNode to the query they run. Shared views and background callers keep the legacy route.

…cle' into pr119468 # Conflicts: # posthog/hogql/test/test_new_events_schema_routing.py

…cle' into pr119468

# Conflicts: # posthog/hogql_queries/query_runner.py

…users The user flag unwrapped every source, so runnable actor wrappers such as FunnelsActorsQuery were opted in and then refused by their runner. It now unwraps only the UI wrappers process_query_model unwraps, and asks native_route_can_run whether the execution mode can use the native route.

…m the request user Both entrypoints pass the request user instead of computing modifiers, so a saved insight's query is parsed once. The flag reads the request's access method itself, so personal API key traffic stays legacy on both the query and insights paths. The flag uses posthog_feature_flag_enabled like other server-side gates.

With the async gate off, cache-only reads took the native cache entry while the async fills they waited on wrote the legacy one, so a flagged user's dashboard tiles could come back empty. The flag now waits for CLICKHOUSE_NATIVE_EVENTS_ASYNC, so every mode agrees on one route. Native stores skip the warming freshness index, which only tracks legacy entries. Entrypoints pass interactive_request instead of the user twice, and the opt-in resolves next to the runner, cheap checks first.

vdekrijger · 2026-10-22T16:38:34Z

@PostHog review

posthog · 2026-10-22T16:38:39Z

@vdekrijger PostHog Review is reviewing this pull request.

posthog

PostHog Review · Standard · reviewed 4005153: no findings.

vdekrijger · 2026-10-27T09:38:26Z

Superseded by #122489, which merged on Oct 26 and routes events_json reads to the events cluster with one rule instead of the per-query opt-in this PR carried. Closing; the branch stays.

vdekrijger self-assigned this Oct 19, 2026

posthog Bot reviewed Oct 19, 2026

View reviewed changes

Comment thread posthog/api/services/query.py Resolved

Comment thread posthog/api/services/query.py Resolved

Comment thread posthog/api/services/query.py Resolved

tests-posthog Bot and others added 2 commits October 19, 2026 11:10

test(mcp): update unit test snapshots

56761a5

Merge remote-tracking branch 'origin/codex/native-events-async-lifecy…

9871a33

…cle' into vasco/native-events-user-flag

vdekrijger added this pull request to stack #116907 October 20, 2026 07:47

vdekrijger added 6 commits October 20, 2026 10:47

Merge branch 'codex/native-events-async-lifecycle' into vasco/native-…

591f45b

…events-user-flag

Merge branch 'codex/native-events-async-lifecycle' into vasco/native-…

c39eb66

…events-user-flag

Merge branch 'codex/native-events-async-lifecycle' into vasco/native-…

dc7fda2

…events-user-flag

Merge branch 'codex/native-events-async-lifecycle' into vasco/native-…

ba295f8

…events-user-flag

Merge branch 'codex/native-events-async-lifecycle' into vasco/native-…

5fc8e14

…events-user-flag

Merge branch 'codex/native-events-async-lifecycle' into vasco/native-…

2187883

…events-user-flag

vdekrijger removed this pull request from stack #116907 October 20, 2026 13:37

vdekrijger mentioned this pull request Oct 20, 2026

feat(hogql): extend native cluster comparisons to typed queries #116838

Closed

vdekrijger added this pull request to stack #120301 October 20, 2026 13:40

tests-posthog Bot and others added 5 commits October 20, 2026 13:45

test(mcp): update unit test snapshots

4b51ee0

Merge branch 'codex/native-events-async-lifecycle' into vasco/native-…

9c1efc0

…events-user-flag

Merge remote-tracking branch 'origin/vasco/native-events-user-flag' i…

11415e1

…nto vasco/native-events-user-flag

Merge branch 'codex/native-events-async-lifecycle' into vasco/native-…

84c3ab0

…events-user-flag

Merge branch 'codex/native-events-async-lifecycle' into vasco/native-…

388d983

…events-user-flag

vdekrijger added the posthog-review Run a Standard PostHog Review label Oct 20, 2026

posthog Bot reviewed Oct 20, 2026

View reviewed changes

posthog Bot removed the posthog-review Run a Standard PostHog Review label Oct 20, 2026

vdekrijger added 2 commits October 20, 2026 16:43

Merge branch 'codex/native-events-async-lifecycle' into vasco/native-…

d23bd40

…events-user-flag

Merge branch 'codex/native-events-async-lifecycle' into vasco/native-…

57c25da

…events-user-flag

vdekrijger added the posthog-review Run a Standard PostHog Review label Oct 20, 2026

posthog Bot reviewed Oct 20, 2026

View reviewed changes

vdekrijger added 7 commits October 21, 2026 13:23

refactor(query): use the shared explicit-route rule in the native rol…

5438bc3

…lout flag

Merge branch 'codex/native-events-async-lifecycle' into vasco/native-…

538e1cc

…events-user-flag

Merge branch 'codex/native-events-async-lifecycle' into vasco/native-…

7553c83

…events-user-flag # Conflicts: # posthog/api/services/query.py

Merge branch 'codex/native-events-async-lifecycle' into vasco/native-…

d2a1ad6

…events-user-flag

Merge branch 'pr115778' into pr119468

23b3dd7

Merge branch 'codex/native-events-async-lifecycle' into vasco/native-…

e1cd102

…events-user-flag

vdekrijger removed this pull request from stack #120301 October 22, 2026 13:25

vdekrijger added this pull request to stack #122087 October 22, 2026 13:26

vdekrijger added 7 commits October 22, 2026 15:32

Merge remote-tracking branch 'origin/codex/native-events-async-lifecy…

81f50ad

…cle' into pr119468 # Conflicts: # posthog/hogql/test/test_new_events_schema_routing.py

Merge remote-tracking branch 'origin/codex/native-events-async-lifecy…

1f292ac

…cle' into pr119468

Merge branch 'pr115778' into pr119468

18fc800

# Conflicts: # posthog/hogql_queries/query_runner.py

Merge branch 'pr115778' into pr119468

071cff7

Merge branch 'pr115778' into pr119468

d5f6f34

This was referenced Oct 22, 2026

feat(clickhouse): support native events query lifecycle #115778

Closed

feat(precompute): build listed projects' precompute from events_json #122031

Closed

vdekrijger added 6 commits October 22, 2026 18:02

Merge branch 'pr115778' into pr119468

c87d86e

Merge branch 'pr115778' into pr119468

6201bc2

Merge branch 'pr115778' into pr119468

e5a7a33

refactor(query-cache): guard the freshness index update once

5639c71

Merge branch 'pr115778' into pr119468

4005153

posthog Bot reviewed Oct 22, 2026

View reviewed changes

aspicer mentioned this pull request Oct 23, 2026

feat(hogql): staff switch for the native events table #122489

Merged

vdekrijger closed this Oct 27, 2026

Navigation Menu

Repository navigation

feat(hogql): opt flagged users into the native events cluster - #119468

feat(hogql): opt flagged users into the native events cluster#119468
vdekrijger wants to merge 56 commits into
codex/native-events-async-lifecyclefrom
vasco/native-events-user-flag

vdekrijger commented Oct 19, 2026 •

edited

Loading

Uh oh!

posthog Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Oct 19, 2026 •

edited

Loading

Snapshots: MCP unit test snapshots updated

Uh oh!

trunk-io Bot commented Oct 19, 2026 •

edited

Loading

Uh oh!

posthog Bot left a comment

Uh oh!

posthog Bot left a comment

Uh oh!

vdekrijger commented Oct 22, 2026

Uh oh!

posthog Bot commented Oct 22, 2026

Uh oh!

posthog Bot left a comment

Uh oh!

vdekrijger commented Oct 27, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

vdekrijger commented Oct 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Changes

How did you test this code?

Release status

Docs update

🤖 Agent context

Uh oh!

posthog Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Oct 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🤖 CI report

Snapshots: MCP unit test snapshots updated

Uh oh!

trunk-io Bot commented Oct 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

posthog Bot left a comment

Choose a reason for hiding this comment

Uh oh!

posthog Bot left a comment

Choose a reason for hiding this comment

Uh oh!

vdekrijger commented Oct 22, 2026

Uh oh!

posthog Bot commented Oct 22, 2026

Uh oh!

posthog Bot left a comment

Choose a reason for hiding this comment

Uh oh!

vdekrijger commented Oct 27, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

vdekrijger commented Oct 19, 2026 •

edited

Loading

github-actions Bot commented Oct 19, 2026 •

edited

Loading

trunk-io Bot commented Oct 19, 2026 •

edited

Loading