Is your feature request related to a problem?
The query-insights plugin captures all search queries as generic DSL queries. When SQL/PPL queries are executed through the _plugins/_sql or _plugins/_ppl endpoints, they generate one or more underlying DSL searches, but there is no way to trace these back to the original SQL/PPL statement. Operators cannot identify which SQL/PPL queries are consuming the most resources, causing latency spikes, or need optimization. The Top N Queries view shows only raw DSL bodies, requiring manual correlation.
What solution would you like?
Propagate SQL/PPL query metadata to the query-insights plugin via thread context headers so that:
- Each DSL search generated by SQL/PPL is tagged with x-query-source (sql/ppl), x-original-query (the SQL/PPL text), and x-query-execution-id (a UUID correlating all DSL searches from one SQL/PPL execution)
- SQL/PPL processing phase timings (parse, analyze, plan) are captured via x-query-phases header and exposed alongside the query record
- Headers are registered as task headers via getTaskHeaders() so they propagate through the transport layer to the query-insights listener
- The Calcite execution path (used by PPL) includes phase tracking alongside the legacy path (used by SQL)
What alternatives have you considered?
Pattern-matching the DSL source in the query-insights listener to detect SQL/PPL-generated queries — rejected as fragile, unreliable, and unable to recover the original SQL/PPL text.
Building a separate SQL-specific query tracking system — rejected as duplicative; the existing query-insights infrastructure already provides top-N ranking, storage, and dashboards.
Adding metadata at the REST layer only — rejected because it wouldn't propagate to child search tasks across nodes in a distributed cluster.
Do you have any additional context?
Implementation PR: #5636
Companion PRs in other repos:
Is your feature request related to a problem?
The query-insights plugin captures all search queries as generic DSL queries. When SQL/PPL queries are executed through the _plugins/_sql or _plugins/_ppl endpoints, they generate one or more underlying DSL searches, but there is no way to trace these back to the original SQL/PPL statement. Operators cannot identify which SQL/PPL queries are consuming the most resources, causing latency spikes, or need optimization. The Top N Queries view shows only raw DSL bodies, requiring manual correlation.
What solution would you like?
Propagate SQL/PPL query metadata to the query-insights plugin via thread context headers so that:
What alternatives have you considered?
Pattern-matching the DSL source in the query-insights listener to detect SQL/PPL-generated queries — rejected as fragile, unreliable, and unable to recover the original SQL/PPL text.
Building a separate SQL-specific query tracking system — rejected as duplicative; the existing query-insights infrastructure already provides top-N ranking, storage, and dashboards.
Adding metadata at the REST layer only — rejected because it wouldn't propagate to child search tasks across nodes in a distributed cluster.
Do you have any additional context?
Implementation PR: #5636
Companion PRs in other repos: