[BUG] plugins.ppl.values.max.limit cannot be honored on the Analytics Engine route (no binding for array_agg(DISTINCT x, limit))

[RyanL1997]

What is the bug?

plugins.ppl.values.max.limit cannot currently be honored on the Analytics Engine (composite/parquet) route. The cap is silently ignored today, and simply making the setting reach the planner is not enough — the resulting plan fails to bind on the DataFusion backend and the query returns HTTP 500.

Root cause

The cap is applied at AST-build time by attaching a limit argument to the aggregate:

ppl/src/main/java/org/opensearch/sql/ppl/parser/AstExpressionBuilder.java — visitValuesAggFunctionCall:

int limit = 0; // Default to unlimited
if (astBuilder.getSettings() != null) {
  Integer settingValue = astBuilder.getSettings().getSettingValue(Key.PPL_VALUES_MAX_LIMIT);
  if (settingValue != null) { limit = settingValue; }
}
if (limit > 0) {
  builder.add(new UnresolvedArgument("limit", AstDSL.intLiteral(limit)));
}

That extra argument lowers values(x) to array_agg(DISTINCT x, limit). The analytics-engine backend has no binding for the two-argument form.

Two things stack here:

1. On the AE route the setting never reaches the planner at all (it is not seeded into UnifiedQueryContext.Builder and was not forwarded by RestUnifiedQueryAction), so limit stays 0 and no argument is attached — the cap is silently ignored.
2. Even once the setting does reach the planner, the backend cannot execute the result.

So the cause recorded in Capability.VALUES_LIMIT_NOT_HONORED ("the aggregate rewriter emits no limit") is accurate in effect, and (2) is the blocking half.

How can one reproduce the bug?

Verified against a live single-node cluster with composite-engine, parquet-data-format, analytics-engine, analytics-backend-datafusion, analytics-backend-lucene and the SQL plugin.

# parquet-backed composite index, auto-generated doc ids
curl -XPUT localhost:9200/ae_probe -H 'Content-Type: application/json' -d '{
 "settings":{"index":{"number_of_shards":1,"pluggable.dataformat.enabled":true,
 "pluggable.dataformat":"composite","composite.primary_data_format":"parquet",
 "composite.secondary_data_formats":["lucene"]}},
 "mappings":{"properties":{"name":{"type":"keyword"},"age":{"type":"integer"}}}}'
# bulk-load 6 docs name=n1..n6, then:
curl -XPUT localhost:9200/_cluster/settings -H 'Content-Type: application/json' \
  -d '{"transient":{"plugins.ppl.values.max.limit":3}}'
curl -XPOST localhost:9200/_plugins/_ppl -H 'Content-Type: application/json' \
  -d '{"query":"source=ae_probe | stats values(name) as v"}'

Current behavior — the cap is ignored, all 6 values returned:

6 values: ['n1','n2','n3','n4','n5','n6']

With the setting forwarded into the unified query context (i.e. the naive fix), the same query fails:

{"error":{"reason":"There was internal problem at backend",
          "details":"Internal error [task_id=63]","type":"RuntimeException"},"status":500}

with, in the node log:

[ERROR][o.o.a.e.DefaultPlanExecutor] [analytics-engine] internal error [task_id=63]
java.lang.UnsupportedOperationException: Unable to find binding for call array_agg(DISTINCT $0, $1)

What is the expected behavior?

values() / list() should honor plugins.ppl.values.max.limit on the analytics-engine route, as they do on the default engine.

Fixing this requires backend support for the limited aggregate form (a binding for array_agg(DISTINCT x, n), or an equivalent sort+limit rewrite in PplAggregateCallRewriter) — it cannot be fixed by settings plumbing alone. Once that lands, Key.PPL_VALUES_MAX_LIMIT can be added to RestUnifiedQueryAction.FORWARDED_CLUSTER_SETTINGS and Capability.VALUES_LIMIT_NOT_HONORED (with the @RequiresCapability on CalciteMultiValueStatsIT.testValuesFunctionRespectsConfiguredLimit) removed.

Do you have any additional context?

Found while fixing settings-forwarding gaps on the AE path in #5611. That PR deliberately excludes this key for the reason above — forwarding it would convert a silently-ignored cap into a hard 500 — and documents the exclusion in code so it is not "helpfully" added later.

Related: #5734, #5735.