diff --git a/book/src/drive/ranked-index-examples.md b/book/src/drive/ranked-index-examples.md index 9c75d3b04b4..20798741803 100644 --- a/book/src/drive/ranked-index-examples.md +++ b/book/src/drive/ranked-index-examples.md @@ -33,7 +33,7 @@ SELECT avg(grade) FROM review **This replaced a non-SQL spelling that never shipped.** An earlier draft put the ranking on the right of a `HAVING` clause — `HAVING avg(grade) IN TOP(3)`, with `TOP` / `BOTTOM` / `MAX` / `MIN` as cross-group primitives. It was removed before release rather than deprecated. The deliberate call: SQL conformance beats a bespoke primitive. Every client author already knows `ORDER BY … LIMIT`; nobody knows `IN TOP(n)`, and the two express exactly the same thing. The retired spelling also had a rough edge the SQL one simply does not have — `= MAX` means *every* group tied at the extreme, which a bounded read cannot prove, so `MAX` / `MIN` had to be permanently refused. `ORDER BY DESC LIMIT 1` is positional and has no such ambiguity. -`HAVING` survives as what it is in SQL: a boolean per-group predicate. It is not yet evaluated (every non-empty `having` is `Unsupported`), and it cannot currently be combined with a ranking `ORDER BY` — the ranked executor reads a pre-sorted secondary and has no way to drop groups from the middle of that walk. +`HAVING` survives as what it is in SQL: a boolean per-group predicate — and since protocol v14 it is evaluated. A grouped aggregate carrying exactly one `having` clause that bounds the selected aggregate (`GROUP BY hashtag HAVING count(*) > 100 LIMIT 100`) is served as a value-bounded range read of the same axis secondary the ranking walks, with the same completeness-proving envelope. An `ORDER BY` naming the selected aggregate may ride along to set the walk direction (`HAVING avg(grade) > 80 ORDER BY avg(grade) DESC LIMIT 5` — the best matches first); what a `having` request cannot carry is rank-window pagination (`OFFSET`, `starting_rank`), because a value-bounded page has no rank base — its continuation is "tighten the bound past the last value seen". That continuation steps past *distinct* aggregate values only: if the `LIMIT` cuts inside a tie (several groups sharing the boundary aggregate), keeping the boundary value repeats the same page and moving past it permanently skips the remaining tied groups, so size the limit above the widest expected tie. The grammar's v1 boundaries: one clause only, on the aggregate the select projects, with a contiguous-range operator (`=`, `>`, `>=`, `<`, `<=`, `BETWEEN` variants; `!=` and `IN` are non-contiguous and refused). ## The Restaurants Contract @@ -666,7 +666,7 @@ Everything below is rejected *before* any grovedb work, and most of it is mirror | **`start_at` / `start_after`** — `InvalidLimit` | The cursor names a document id, but a ranked walk iterates an aggregate-ordered keyspace in which document ids do not appear. | | **`order_by` naming anything but the selected aggregate**, or more than one clause — `InvalidParameter` | The single ordering clause *is* the ranking, and the secondary is sorted by one aggregate only. An ordering on the `GROUP BY` property, on an unrelated field, or a second tie-break clause names an order the secondary cannot produce. Accepting and silently ignoring it is the one genuinely dangerous option. Use the aggregate's own name (`$count` for `COUNT(*)`), or flip `ASC` ↔ `DESC` to reverse the ranking. | | **`group_by` with ≠ 1 property** — `InvalidParameter` | Ranked indexes are single-property, so there is no compound grouping to rank over. | -| **any non-empty `having`** — `Unsupported` | `HAVING` is a boolean per-group predicate and is not evaluated at any protocol version. It also cannot combine with a ranking `ORDER BY`: the ranked executor reads a pre-sorted secondary and has no way to drop groups from the middle of that walk. | +| **`having` that isn't one contiguous bound on the selected aggregate** | A grouped single-clause `having` bounding the selected aggregate is **served** since protocol v14 — it routes to the having-range executor, a value-bounded range read of the same axis secondary (see the `HAVING` paragraph above). What stays rejected: multiple clauses (a second predicate needs a per-candidate post-check no executor performs), a clause on a different aggregate than the select projects (same reason), non-contiguous operators (`!=`, `IN`), `having` without `group_by` (a single implicit group is a plain aggregate the client can bound itself), and `OFFSET` / `start_at` alongside `having` (a value-bounded page has no rank base; continuation is by tightening the bound). Protocol v13 and earlier reject every non-empty `having` unchanged. | | **no `order_by` at all, on a grouped aggregate** — routed elsewhere | Without an ordering this is a plain grouped aggregate, not a ranking; the caller wanted the `DocumentSplitCounts` / `DocumentSplitSums` / `DocumentSplitAverages` surface. | | **`COUNT(field)`** (non-`*`) — `Unsupported`; **`SUM` / `AVG` with an empty field** — `InvalidParameter` | The Count axis ranks group cardinality and takes no field; the Sum and Avg axes rank the property the index accumulates and require it. | | **`limit` unset, `0`, or `> 100`** — `InvalidLimit` | A ranking with no `n` has no size, and `LIMIT 0` selects nothing. The ceiling is a **hard limit, not a clamp**, because `k` is echoed in the proof envelope and re-checked by the verifier — a silent clamp would produce a proof the client's own reconstruction rejects. | diff --git a/packages/dapi-grpc/clients/platform/v0/objective-c/Platform.pbobjc.h b/packages/dapi-grpc/clients/platform/v0/objective-c/Platform.pbobjc.h index 90b34897e29..295993f3f5e 100644 --- a/packages/dapi-grpc/clients/platform/v0/objective-c/Platform.pbobjc.h +++ b/packages/dapi-grpc/clients/platform/v0/objective-c/Platform.pbobjc.h @@ -2648,12 +2648,21 @@ typedef GPB_ENUM(GetDocumentsRequest_HavingClause_Right_OneOfCase) { * before release rather than deprecated, because it invented * non-SQL grammar for something SQL already expresses. * - * **`HAVING` cannot yet combine with an aggregate `ORDER BY`.** - * The ranked executor reads a pre-sorted per-axis secondary and - * has no way to drop groups from the middle of that walk, so a - * request carrying both a non-empty `having` and a ranking - * `order_by` is rejected with `Unsupported` rather than served - * with one of the two silently ignored. + * **From protocol v14 a single `HAVING` clause is served as a + * bounded range read** (having-range mode): `SELECT GROUP BY + * p HAVING [ORDER BY ASC|DESC] + * LIMIT n` answers from the same per-axis secondary as ranked + * mode, on an index declaring the matching ranked axis. The + * clause's aggregate must be the selected aggregate, the operator + * must describe one contiguous range (`NOT_EQUAL` / `IN` are + * rejected), and the optional `ORDER BY` picks the walk direction + * using the same order-key spelling as ranked mode: `f` for + * `SUM(f)` / `AVG(f)`, the `$count` sentinel for `COUNT(*)` — + * never an explicit `OrderClause.aggregate` target, which is + * rejected. See the supported-shape table on + * `GetDocumentsRequestV1`. On protocol v13 and earlier every + * non-empty `having` stays rejected with `Unsupported`, exactly as + * before. * * The operator set mirrors `WhereOperator` minus `STARTS_WITH` * (prefix matching has no natural meaning against a scalar @@ -2897,12 +2906,16 @@ typedef GPB_ENUM(GetDocumentsRequest_GetDocumentsRequestV1_Start_OneOfCase) { * It returns `ResultData.ranked`. See `order_by` and the * supported-shape table below. * - * `having` is a boolean per-group predicate and is **still** - * `Unsupported` at every protocol version, ranked mode or not - * (`"HAVING clause is not yet implemented"`). It carries no ranking - * spelling: an earlier draft put cross-group ranking on the right of - * a `HAVING` (`HAVING AVG(grade) IN TOP(5)`) and that grammar was - * removed before release in favour of `ORDER BY` + `LIMIT`. + * **Having-range mode** is served from protocol v14: a single + * `having` clause whose aggregate is the selected aggregate turns + * the request into a bounded range read over the same per-axis + * secondary ranked mode walks, answered in `ResultData.ranked`. + * On protocol v13 and earlier every non-empty `having` is rejected + * (`"HAVING clause is not yet implemented"`). `having` carries no + * ranking spelling: an earlier draft put cross-group ranking on the + * right of a `HAVING` (`HAVING AVG(grade) IN TOP(5)`) and that + * grammar was removed before release in favour of `ORDER BY` + + * `LIMIT`. See the supported-shape table below. * * **Supported shapes** (everything else rejects with a typed * `QuerySyntaxError::Unsupported` so callers can detect un-wired @@ -2931,8 +2944,12 @@ typedef GPB_ENUM(GetDocumentsRequest_GetDocumentsRequestV1_Start_OneOfCase) { * - exactly one `group_by` property, exactly one `order_by` clause naming the select's aggregate (`f` for `SUM(f)` / `AVG(f)`, the `$count` sentinel for `COUNT(*)`), a `limit` in `1 ..= 100`, an optional `offset`, and no `where` / `having` / `start_at`, on an index declaring the matching `rankedCountable` / `rankedSummable` / `rankedAverageable` axis → ranked executor, answered in `ResultData.ranked`. * - `DESC` is the "top n" reading (walk the axis from the largest aggregate down), `ASC` the "bottom n" reading. Worked example: `SELECT AVG(grade) GROUP BY restaurantId ORDER BY grade DESC LIMIT 1 OFFSET 4` is the 5th-best restaurant. * + * `select=, group_by=[p], having=[ ]` (protocol v14+) — **having-range mode**: + * - exactly one `group_by` property, exactly one `having` clause whose aggregate is the select's aggregate, an operator describing one contiguous range (`EQUAL`, `GREATER_THAN[_OR_EQUALS]`, `LESS_THAN[_OR_EQUALS]`, `BETWEEN*`; `NOT_EQUAL` / `IN` rejected), a `limit` in `1 ..= 100`, an optional `order_by` naming the same aggregate (walk direction; ascending by default), and no `where` / `offset` / `start_at` / `start_after`, on an index declaring the matching ranked axis → having-range executor, answered in `ResultData.ranked`. + * - no offset or cursor pagination: a page cut at `limit` continues only by tightening the bound past the last *distinct* aggregate value seen; a cut inside a tie (several groups sharing the boundary aggregate) cannot be continued, so size `limit` above the widest expected tie. + * * **Rejected shapes** (return `Unsupported`): - * - any non-empty `having`, at every protocol version. + * - any non-empty `having` on protocol v13 and earlier; at v14+, any `having` shape outside having-range mode above (multiple clauses, an aggregate other than the select's, `NOT_EQUAL` / `IN`, or a carried `where` / `offset` / cursor). * - at v14+: a ranked-shaped request carrying a `where` clause, a `start_at` / `start_after` cursor, more than one `order_by`, or an `order_by` naming anything but the selected aggregate. * - `select=DOCUMENTS` with non-empty `group_by`. * - `select=COUNT` with `group_by` on a field that is not constrained by an `In` or range where clause. @@ -3110,12 +3127,13 @@ GPB_FINAL @interface GetDocumentsRequest_GetDocumentsRequestV1 : GPBMessage * `HavingClause` / `HavingAggregate` for the operator and * aggregate-function catalogs. * - * **Every non-empty `having` is rejected**, at every protocol - * version, with `Unsupported("HAVING clause is not yet - * implemented")`. The wire shape ships ahead of evaluation so - * callers can construct full `HAVING COUNT(*) > 5 AND - * SUM(amount) > 100` requests in their builders, and so the - * capability can land without another version bump. + * **From protocol v14 a single clause is served** as a bounded + * range read — having-range mode; see the message-level + * supported-shape table. On v13 and earlier every non-empty + * `having` is rejected with `Unsupported("HAVING clause is not + * yet implemented")`. Multi-clause `HAVING COUNT(*) > 5 AND + * SUM(amount) > 100` requests can still be constructed on the + * wire, but stay rejected until a multi-clause evaluator lands. * * **`having` does not express ranking.** "The n highest-scoring * groups" is `ORDER BY DESC LIMIT n` diff --git a/packages/dapi-grpc/protos/platform/v0/platform.proto b/packages/dapi-grpc/protos/platform/v0/platform.proto index c93f36b0a19..bfa0b5cf68b 100644 --- a/packages/dapi-grpc/protos/platform/v0/platform.proto +++ b/packages/dapi-grpc/protos/platform/v0/platform.proto @@ -703,12 +703,21 @@ message GetDocumentsRequest { // before release rather than deprecated, because it invented // non-SQL grammar for something SQL already expresses. // - // **`HAVING` cannot yet combine with an aggregate `ORDER BY`.** - // The ranked executor reads a pre-sorted per-axis secondary and - // has no way to drop groups from the middle of that walk, so a - // request carrying both a non-empty `having` and a ranking - // `order_by` is rejected with `Unsupported` rather than served - // with one of the two silently ignored. + // **From protocol v14 a single `HAVING` clause is served as a + // bounded range read** (having-range mode): `SELECT GROUP BY + // p HAVING [ORDER BY ASC|DESC] + // LIMIT n` answers from the same per-axis secondary as ranked + // mode, on an index declaring the matching ranked axis. The + // clause's aggregate must be the selected aggregate, the operator + // must describe one contiguous range (`NOT_EQUAL` / `IN` are + // rejected), and the optional `ORDER BY` picks the walk direction + // using the same order-key spelling as ranked mode: `f` for + // `SUM(f)` / `AVG(f)`, the `$count` sentinel for `COUNT(*)` — + // never an explicit `OrderClause.aggregate` target, which is + // rejected. See the supported-shape table on + // `GetDocumentsRequestV1`. On protocol v13 and earlier every + // non-empty `having` stays rejected with `Unsupported`, exactly as + // before. // // The operator set mirrors `WhereOperator` minus `STARTS_WITH` // (prefix matching has no natural meaning against a scalar @@ -849,12 +858,16 @@ message GetDocumentsRequest { // It returns `ResultData.ranked`. See `order_by` and the // supported-shape table below. // - // `having` is a boolean per-group predicate and is **still** - // `Unsupported` at every protocol version, ranked mode or not - // (`"HAVING clause is not yet implemented"`). It carries no ranking - // spelling: an earlier draft put cross-group ranking on the right of - // a `HAVING` (`HAVING AVG(grade) IN TOP(5)`) and that grammar was - // removed before release in favour of `ORDER BY` + `LIMIT`. + // **Having-range mode** is served from protocol v14: a single + // `having` clause whose aggregate is the selected aggregate turns + // the request into a bounded range read over the same per-axis + // secondary ranked mode walks, answered in `ResultData.ranked`. + // On protocol v13 and earlier every non-empty `having` is rejected + // (`"HAVING clause is not yet implemented"`). `having` carries no + // ranking spelling: an earlier draft put cross-group ranking on the + // right of a `HAVING` (`HAVING AVG(grade) IN TOP(5)`) and that + // grammar was removed before release in favour of `ORDER BY` + + // `LIMIT`. See the supported-shape table below. // // **Supported shapes** (everything else rejects with a typed // `QuerySyntaxError::Unsupported` so callers can detect un-wired @@ -883,8 +896,12 @@ message GetDocumentsRequest { // - exactly one `group_by` property, exactly one `order_by` clause naming the select's aggregate (`f` for `SUM(f)` / `AVG(f)`, the `$count` sentinel for `COUNT(*)`), a `limit` in `1 ..= 100`, an optional `offset`, and no `where` / `having` / `start_at`, on an index declaring the matching `rankedCountable` / `rankedSummable` / `rankedAverageable` axis → ranked executor, answered in `ResultData.ranked`. // - `DESC` is the "top n" reading (walk the axis from the largest aggregate down), `ASC` the "bottom n" reading. Worked example: `SELECT AVG(grade) GROUP BY restaurantId ORDER BY grade DESC LIMIT 1 OFFSET 4` is the 5th-best restaurant. // + // `select=, group_by=[p], having=[ ]` (protocol v14+) — **having-range mode**: + // - exactly one `group_by` property, exactly one `having` clause whose aggregate is the select's aggregate, an operator describing one contiguous range (`EQUAL`, `GREATER_THAN[_OR_EQUALS]`, `LESS_THAN[_OR_EQUALS]`, `BETWEEN*`; `NOT_EQUAL` / `IN` rejected), a `limit` in `1 ..= 100`, an optional `order_by` naming the same aggregate (walk direction; ascending by default), and no `where` / `offset` / `start_at` / `start_after`, on an index declaring the matching ranked axis → having-range executor, answered in `ResultData.ranked`. + // - no offset or cursor pagination: a page cut at `limit` continues only by tightening the bound past the last *distinct* aggregate value seen; a cut inside a tie (several groups sharing the boundary aggregate) cannot be continued, so size `limit` above the widest expected tie. + // // **Rejected shapes** (return `Unsupported`): - // - any non-empty `having`, at every protocol version. + // - any non-empty `having` on protocol v13 and earlier; at v14+, any `having` shape outside having-range mode above (multiple clauses, an aggregate other than the select's, `NOT_EQUAL` / `IN`, or a carried `where` / `offset` / cursor). // - at v14+: a ranked-shaped request carrying a `where` clause, a `start_at` / `start_after` cursor, more than one `order_by`, or an `order_by` naming anything but the selected aggregate. // - `select=DOCUMENTS` with non-empty `group_by`. // - `select=COUNT` with `group_by` on a field that is not constrained by an `In` or range where clause. @@ -1084,12 +1101,13 @@ message GetDocumentsRequest { // `HavingClause` / `HavingAggregate` for the operator and // aggregate-function catalogs. // - // **Every non-empty `having` is rejected**, at every protocol - // version, with `Unsupported("HAVING clause is not yet - // implemented")`. The wire shape ships ahead of evaluation so - // callers can construct full `HAVING COUNT(*) > 5 AND - // SUM(amount) > 100` requests in their builders, and so the - // capability can land without another version bump. + // **From protocol v14 a single clause is served** as a bounded + // range read — having-range mode; see the message-level + // supported-shape table. On v13 and earlier every non-empty + // `having` is rejected with `Unsupported("HAVING clause is not + // yet implemented")`. Multi-clause `HAVING COUNT(*) > 5 AND + // SUM(amount) > 100` requests can still be constructed on the + // wire, but stay rejected until a multi-clause evaluator lands. // // **`having` does not express ranking.** "The n highest-scoring // groups" is `ORDER BY DESC LIMIT n` diff --git a/packages/rs-drive-abci/src/query/document_query/v1/compute_aggregate_mode_and_check_limit/mod.rs b/packages/rs-drive-abci/src/query/document_query/v1/compute_aggregate_mode_and_check_limit/mod.rs index 8fe31b340b8..4b5eb65758b 100644 --- a/packages/rs-drive-abci/src/query/document_query/v1/compute_aggregate_mode_and_check_limit/mod.rs +++ b/packages/rs-drive-abci/src/query/document_query/v1/compute_aggregate_mode_and_check_limit/mod.rs @@ -5,11 +5,13 @@ //! against the v1 query surface is executed: as one of the four //! `(group_by × where)` grouped modes, or — from feature version 1 — //! as a *ranked* request when the request orders by the aggregate it -//! selects (`GROUP BY p ORDER BY `). Routing -//! here only asks whether the `order_by` names that aggregate; the -//! direction, the limit's bounds and the offset are drive's call and -//! are made in `detect_ranked_mode`. It also enforces the per-mode -//! `accepts_limit()` contract on the grouped path. +//! selects (`GROUP BY p ORDER BY `), or — from +//! feature version 2 — as a *having-range* request when a grouped +//! aggregate carries exactly one `having` clause. Routing here only +//! asks which shape the request has; the direction, the bounds, the +//! limit's contract and the offset are drive's call and are made in +//! `detect_ranked_mode` / `detect_having_mode`. It also enforces the +//! per-mode `accepts_limit()` contract on the grouped path. //! //! The routing rules it embeds are part of the query contract clients //! see on the wire — a change to which `(group_by × where_clauses × @@ -17,10 +19,10 @@ //! dispatcher runs on every v1 query request. Versioning it lets later //! protocol bumps adjust the routing table without breaking older //! nodes' replay of historical traffic, and is what keeps a -//! mixed-version network in agreement across the ranked-query -//! activation: protocol version 13 and earlier select v0, which has no -//! ranked path at all, while protocol version 14 selects v1 and answers -//! ranked queries. +//! mixed-version network in agreement across the ranked-query and +//! having-range activations: protocol version 13 and earlier select v0, +//! which has neither path, while protocol version 14 selects v2 and +//! answers both. //! //! Lives next to the v1 query handler (the only call site today) and //! is dispatched via the `DriveAbciDocumentQueryHelperVersions` slot @@ -28,6 +30,7 @@ mod v0; mod v1; +mod v2; use crate::error::query::QueryError; use dpp::version::PlatformVersion; @@ -56,6 +59,15 @@ pub(super) enum AggregateRouting { Grouped(CountMode), /// Ranked aggregate: execute through the ranked (top-k) surface. Ranked, + /// Boolean-`HAVING` range: a grouped aggregate carrying exactly one + /// `having` clause, executed through + /// `Drive::execute_document_having_request` as a value-bounded range + /// read of the covering ranked index's axis secondary. The + /// feature-version-2 addition. Carries no data for the same reason + /// `Ranked` carries none: drive owns the having grammar + /// (`detect_having_mode`), and routing only decides *where* the + /// request goes. + HavingRange, } /// Decide how a `SELECT COUNT` / `SUM` / `AVG` request executes. @@ -113,10 +125,19 @@ pub(super) fn compute_aggregate_mode_and_check_limit( having, function_name, ), + 2 => v2::compute_aggregate_mode_and_check_limit_v2( + select, + group_by, + where_clauses, + order_by, + limit, + having, + function_name, + ), version => Err(QueryError::Drive(drive::error::Error::Drive( drive::error::drive::DriveError::UnknownVersionMismatch { method: "compute_aggregate_mode_and_check_limit".to_string(), - known_versions: vec![0, 1], + known_versions: vec![0, 1, 2], received: version, }, ))), diff --git a/packages/rs-drive-abci/src/query/document_query/v1/compute_aggregate_mode_and_check_limit/v2/mod.rs b/packages/rs-drive-abci/src/query/document_query/v1/compute_aggregate_mode_and_check_limit/v2/mod.rs new file mode 100644 index 00000000000..39151e09f28 --- /dev/null +++ b/packages/rs-drive-abci/src/query/document_query/v1/compute_aggregate_mode_and_check_limit/v2/mod.rs @@ -0,0 +1,63 @@ +//! Feature version 2 of the aggregate routing helper: the boolean-`HAVING` +//! range activation. +//! +//! Differs from v1 in exactly one branch: a grouped aggregate carrying a +//! **single** `having` clause routes to the having-range executor +//! ([`AggregateRouting::HavingRange`]) instead of being refused. Routing +//! here only asks "is there a grouped select with exactly one having +//! clause?" — whether the clause bounds the selected aggregate, whether +//! its operator translates to a contiguous range, whether an `order_by` +//! is compatible, and the limit's bounds are all drive's call, made in +//! `detect_having_mode`; a routing-layer copy of that grammar could +//! disagree with drive's after a version bump. +//! +//! Multi-clause `having` (implicit AND) keeps the `not_yet_implemented` +//! contract: each extra clause needs a per-candidate post-check against +//! the primary that no executor performs yet. And `having` without +//! `group_by` still falls through to the v1 → v0 blanket rejection — a +//! global aggregate produces one row, and bounding it is a client-side +//! comparison, not a query. + +use super::v1::compute_aggregate_mode_and_check_limit_v1; +use super::AggregateRouting; +use crate::error::query::QueryError; +use crate::query::document_query::v1::not_yet_implemented; +use drive::query::{HavingClause, OrderClause, SelectProjection, WhereClause}; + +#[allow(clippy::too_many_arguments)] +pub(super) fn compute_aggregate_mode_and_check_limit_v2( + select: &SelectProjection, + group_by: &[String], + where_clauses: &[WhereClause], + order_by: &[OrderClause], + limit: Option, + having: &[HavingClause], + function_name: &str, +) -> Result { + if !having.is_empty() && !group_by.is_empty() { + return match having { + [_single] => Ok(AggregateRouting::HavingRange), + many => Err(not_yet_implemented(&format!( + "multiple HAVING clauses (implicit AND): got {}. One clause on the \ + selected {function_name} aggregate is served as a single contiguous \ + range read of the covering ranked index's axis secondary; additional \ + clauses would need a per-candidate post-check that is not implemented. \ + Narrow to a single clause", + many.len() + ))), + }; + } + + // No having (or no group_by, where a having still dies in v0's + // blanket rejection): identical routing to v1, including its ranked + // detection and its delegation to v0 for non-ranked shapes. + compute_aggregate_mode_and_check_limit_v1( + select, + group_by, + where_clauses, + order_by, + limit, + having, + function_name, + ) +} diff --git a/packages/rs-drive-abci/src/query/document_query/v1/dispatch/average.rs b/packages/rs-drive-abci/src/query/document_query/v1/dispatch/average.rs new file mode 100644 index 00000000000..33a53487d65 --- /dev/null +++ b/packages/rs-drive-abci/src/query/document_query/v1/dispatch/average.rs @@ -0,0 +1,241 @@ +//! `RoutingDecision::Average` — the grouped average surface. + +use super::super::not_yet_implemented; +use crate::error::query::QueryError; +use crate::error::Error; +use crate::platform_types::platform::Platform; +use crate::platform_types::platform_state::PlatformState; +use crate::query::response_metadata::CheckpointUsed; +use crate::query::QueryValidationResult; +use dapi_grpc::platform::v0::get_documents_request::get_documents_request_v1::Start as RequestV1Start; +use dapi_grpc::platform::v0::get_documents_response::get_documents_response_v1::{ + average_results, result_data, AverageAggregate, AverageEntries, AverageEntry, AverageResults, + ResultData, +}; +use dapi_grpc::platform::v0::get_documents_response::{ + get_documents_response_v1, GetDocumentsResponseV1, +}; +use dpp::check_validation_result_with_data; +use dpp::data_contract::accessors::v0::DataContractV0Getters; +use dpp::identifier::Identifier; +use dpp::validation::ValidationResult; +use dpp::version::PlatformVersion; +use drive::error::query::QuerySyntaxError; +use drive::query::{ + AverageEntry as DriveAverageEntry, AverageMode, CountMode, DocumentAverageRequest, + DocumentAverageResponse, OrderClause, WhereClause, +}; +use drive::util::grove_operations::GroveDBToUse; + +impl Platform { + /// Dispatch a `select = AVG(field)` request to + /// [`Drive::execute_document_average_request`] and map the response + /// into a `GetDocumentsResponseV1` carrying an `AverageResults` + /// payload (or a `Proof` payload when prove=true). + /// + /// Parallels [`Self::dispatch_sum_v1`] line-by-line — same request + /// construction, same error → typed-rejection mapping, same prove + /// vs no-prove split. The response shape mapping differs: + /// `DocumentAverageResponse::Aggregate { count, sum }` → + /// `AverageResults::aggregate_average`, + /// `DocumentAverageResponse::Entries(_)` → `AverageResults::entries`, + /// `DocumentAverageResponse::Proof(_)` → outer `result.proof`. + #[allow(clippy::too_many_arguments)] + pub(in crate::query::document_query::v1) fn dispatch_average_v1( + &self, + data_contract_id: Vec, + document_type_name: String, + where_clauses: Vec, + order_clauses: Vec, + limit: Option, + start: Option, + prove: bool, + sum_property: String, + mode: CountMode, + platform_state: &PlatformState, + platform_version: &PlatformVersion, + ) -> Result, Error> { + if start.is_some() { + return Ok(QueryValidationResult::new_with_error(not_yet_implemented( + "start_after / start_at with SELECT AVG (paginate by narrowing the \ + range clause itself)", + ))); + } + + let contract_id: Identifier = + check_validation_result_with_data!(data_contract_id.try_into().map_err(|_| { + QueryError::InvalidArgument( + "id must be a valid identifier (32 bytes long)".to_string(), + ) + })); + + let (_, contract_fetch_info) = self.drive.get_contract_with_fetch_info_and_fee( + contract_id.to_buffer(), + None, + true, + None, + platform_version, + )?; + let contract_fetch_info = check_validation_result_with_data!(contract_fetch_info.ok_or( + QueryError::Query(QuerySyntaxError::DataContractNotFound( + "contract not found when querying from value with contract info", + )) + )); + let contract_ref = &contract_fetch_info.contract; + let document_type = check_validation_result_with_data!(contract_ref + .document_type_for_name(document_type_name.as_str()) + .map_err(|_| QueryError::InvalidArgument(format!( + "document type {} not found for contract {}", + document_type_name, contract_id + )))); + + // `AverageMode` mirrors `CountMode` 1:1 — map across. + let avg_mode = match mode { + CountMode::Aggregate => AverageMode::Aggregate, + CountMode::GroupByIn => AverageMode::GroupByIn, + CountMode::GroupByRange => AverageMode::GroupByRange, + CountMode::GroupByCompound => AverageMode::GroupByCompound, + }; + + let drive_request = DocumentAverageRequest { + contract: contract_ref, + document_type, + sum_property, + where_clauses, + order_clauses, + mode: avg_mode, + limit, + prove, + drive_config: &self.config.drive, + }; + let drive_response = + match self + .drive + .execute_document_average_request(drive_request, None, platform_version) + { + Ok(r) => r, + Err(drive::error::Error::Query(qe)) => { + return Ok(QueryValidationResult::new_with_error(QueryError::Query(qe))); + } + Err(e) => return Err(e.into()), + }; + + let response = match drive_response { + DocumentAverageResponse::Aggregate { count, sum } => GetDocumentsResponseV1 { + result: Some(get_documents_response_v1::Result::Data(ResultData { + variant: Some(result_data::Variant::Averages(AverageResults { + variant: Some(average_results::Variant::AggregateAverage( + AverageAggregate { count, sum }, + )), + })), + })), + metadata: Some(self.response_metadata_v0(platform_state, CheckpointUsed::Current)), + }, + DocumentAverageResponse::Entries(entries) => { + if avg_mode == AverageMode::Aggregate { + // Mirror sum-side's fold for the `select=AVG, + // group_by=[]` + PerInValue executor combo. Fold + // both count and sum across In branches. Either + // axis overflowing is surfaced as a typed + // `QuerySyntaxError::Unsupported` so the client + // doesn't get a silently-saturated answer to + // divide against (which would also misreport the + // average). + let mut total_count: u64 = 0; + let mut total_sum: i64 = 0; + let mut overflow_axis: Option<&'static str> = None; + for e in &entries { + match total_count.checked_add(e.count.unwrap_or(0)) { + Some(c) => total_count = c, + None => { + overflow_axis = Some("count"); + break; + } + } + match total_sum.checked_add(e.sum.unwrap_or(0)) { + Some(s) => total_sum = s, + None => { + overflow_axis = Some("sum"); + break; + } + } + } + if let Some(axis) = overflow_axis { + return Ok(QueryValidationResult::new_with_error(QueryError::Query( + QuerySyntaxError::Unsupported(format!( + "aggregate AVG across In branches overflows {axis} \ + ({} axis range); narrow the In set or query branches \ + individually", + if axis == "count" { "u64" } else { "i64" }, + )), + ))); + } + GetDocumentsResponseV1 { + result: Some(get_documents_response_v1::Result::Data(ResultData { + variant: Some(result_data::Variant::Averages(AverageResults { + variant: Some(average_results::Variant::AggregateAverage( + AverageAggregate { + count: total_count, + sum: total_sum, + }, + )), + })), + })), + metadata: Some( + self.response_metadata_v0(platform_state, CheckpointUsed::Current), + ), + } + } else { + GetDocumentsResponseV1 { + result: Some(get_documents_response_v1::Result::Data(ResultData { + variant: Some(result_data::Variant::Averages(AverageResults { + variant: Some(average_results::Variant::Entries(AverageEntries { + entries: entries + .into_iter() + .map(into_v1_average_entry) + .collect(), + })), + })), + })), + metadata: Some( + self.response_metadata_v0(platform_state, CheckpointUsed::Current), + ), + } + } + } + DocumentAverageResponse::Proof(proof_bytes) => { + let (grovedb_used, proof) = + self.response_proof_v0(platform_state, proof_bytes, GroveDBToUse::Current)?; + GetDocumentsResponseV1 { + result: Some(get_documents_response_v1::Result::Proof(proof)), + metadata: Some(self.response_metadata_v0(platform_state, grovedb_used)), + } + } + }; + + Ok(QueryValidationResult::new_with_data(response)) + } +} + +/// Translate an rs-drive `AverageEntry` into the wire `AverageEntry`. +/// Mirror of [`into_v1_entry`] + [`into_v1_sum_entry`] for the average +/// surface (carries both count and sum so the client can divide). +/// +/// `zip_entries` in `drive_document_average_query::drive_dispatcher` +/// performs a strict two-pointer merge that errors out as +/// `CorruptedCodeExecution` on any per-`(in_key, key)` divergence +/// between the count and sum streams. So by the time an entry reaches +/// this mapper, both axes have already been asserted to agree on +/// `Some`-vs-`None` for the same key — meaning the dangerous +/// `(count: None, sum: Some(V))` bucket that could let a client +/// divide V by 0 cannot exist. The `unwrap_or(0)` below is therefore +/// defense-in-depth (same as [`into_v1_entry`] / [`into_v1_sum_entry`] +/// for individual count / sum entries) rather than load-bearing. +fn into_v1_average_entry(e: DriveAverageEntry) -> AverageEntry { + AverageEntry { + in_key: e.in_key, + key: e.key, + count: e.count.unwrap_or(0), + sum: e.sum.unwrap_or(0), + } +} diff --git a/packages/rs-drive-abci/src/query/document_query/v1/dispatch/count.rs b/packages/rs-drive-abci/src/query/document_query/v1/dispatch/count.rs new file mode 100644 index 00000000000..7af8f56ef41 --- /dev/null +++ b/packages/rs-drive-abci/src/query/document_query/v1/dispatch/count.rs @@ -0,0 +1,213 @@ +//! `RoutingDecision::Count` — the count surfaces (aggregate and +//! per-group entries). + +use super::super::not_yet_implemented; +use crate::error::query::QueryError; +use crate::error::Error; +use crate::platform_types::platform::Platform; +use crate::platform_types::platform_state::PlatformState; +use crate::query::response_metadata::CheckpointUsed; +use crate::query::QueryValidationResult; +use dapi_grpc::platform::v0::get_documents_request::get_documents_request_v1::Start as RequestV1Start; +use dapi_grpc::platform::v0::get_documents_response::get_documents_response_v1::{ + count_results, result_data, CountEntries, CountEntry, CountResults, ResultData, +}; +use dapi_grpc::platform::v0::get_documents_response::{ + get_documents_response_v1, GetDocumentsResponseV1, +}; +use dpp::check_validation_result_with_data; +use dpp::data_contract::accessors::v0::DataContractV0Getters; +use dpp::identifier::Identifier; +use dpp::validation::ValidationResult; +use dpp::version::PlatformVersion; +use drive::error::query::QuerySyntaxError; +use drive::query::{ + CountMode, DocumentCountRequest, DocumentCountResponse, OrderClause, SplitCountEntry, + WhereClause, +}; +use drive::util::grove_operations::GroveDBToUse; + +impl Platform { + /// Forward a `select = COUNT` request to drive's count + /// dispatcher. `mode` is the SQL-shape contract derived from + /// `(select, group_by, where)` by `validate_and_route`; drive + /// uses it to pick the executor strategy and decide whether to + /// collapse the response to a single aggregate or return per- + /// group entries. The wire response is `GetDocumentsResponseV1` + /// with the inner `ResultData.counts` variant for non-proof + /// results. + #[allow(clippy::too_many_arguments)] + pub(in crate::query::document_query::v1) fn dispatch_count_v1( + &self, + data_contract_id: Vec, + document_type_name: String, + where_clauses: Vec, + order_clauses: Vec, + limit: Option, + start: Option, + prove: bool, + mode: CountMode, + platform_state: &PlatformState, + platform_version: &PlatformVersion, + ) -> Result, Error> { + if start.is_some() { + return Ok(QueryValidationResult::new_with_error(not_yet_implemented( + "start_after / start_at with SELECT COUNT (paginate by narrowing the \ + range clause itself)", + ))); + } + + let contract_id: Identifier = + check_validation_result_with_data!(data_contract_id.try_into().map_err(|_| { + QueryError::InvalidArgument( + "id must be a valid identifier (32 bytes long)".to_string(), + ) + })); + + let (_, contract_fetch_info) = self.drive.get_contract_with_fetch_info_and_fee( + contract_id.to_buffer(), + None, + true, + None, + platform_version, + )?; + let contract_fetch_info = check_validation_result_with_data!(contract_fetch_info.ok_or( + QueryError::Query(QuerySyntaxError::DataContractNotFound( + "contract not found when querying from value with contract info", + )) + )); + let contract_ref = &contract_fetch_info.contract; + let document_type = check_validation_result_with_data!(contract_ref + .document_type_for_name(document_type_name.as_str()) + .map_err(|_| QueryError::InvalidArgument(format!( + "document type {} not found for contract {}", + document_type_name, contract_id + )))); + + let drive_request = DocumentCountRequest { + contract: contract_ref, + document_type, + where_clauses, + order_clauses, + mode, + limit, + prove, + drive_config: &self.config.drive, + }; + let drive_response = + match self + .drive + .execute_document_count_request(drive_request, None, platform_version) + { + Ok(r) => r, + Err(drive::error::Error::Query(qe)) => { + return Ok(QueryValidationResult::new_with_error(QueryError::Query(qe))); + } + Err(e) => return Err(e.into()), + }; + + let response = match drive_response { + DocumentCountResponse::Aggregate(count) => GetDocumentsResponseV1 { + result: Some(get_documents_response_v1::Result::Data(ResultData { + variant: Some(result_data::Variant::Counts(CountResults { + variant: Some(count_results::Variant::AggregateCount(count)), + })), + })), + metadata: Some(self.response_metadata_v0(platform_state, CheckpointUsed::Current)), + }, + DocumentCountResponse::Entries(entries) => { + if mode.is_aggregate() { + // `select=COUNT, group_by=[]` against a request + // that drove a PerInValue execution (In + no + // range + no prove). Sum entries into a single + // aggregate before emission. `checked_add` + // surfaces u64 overflow as a typed + // `QuerySyntaxError::Unsupported`; realistic + // ceiling is `|In| × max_per-branch-count` (well + // under u64), so triggering this path requires + // either a misconfigured count tree or an + // executor bug. + let mut total: u64 = 0; + let mut overflow = false; + for e in &entries { + // `count.unwrap_or(0)` here is safe: this + // arm is server-side, summing entries the + // executor emitted. Executor never emits + // `None` (that's an SDK-side + // synthesis-for-missing concept). The + // `unwrap_or(0)` is a belt-and-suspenders + // guard against any future executor that + // forgets the contract. + match total.checked_add(e.count.unwrap_or(0)) { + Some(t) => total = t, + None => { + overflow = true; + break; + } + } + } + if overflow { + return Ok(QueryValidationResult::new_with_error(QueryError::Query( + QuerySyntaxError::Unsupported( + "aggregate COUNT across In branches overflows u64 — \ + narrow the In set or query branches individually" + .to_string(), + ), + ))); + } + GetDocumentsResponseV1 { + result: Some(get_documents_response_v1::Result::Data(ResultData { + variant: Some(result_data::Variant::Counts(CountResults { + variant: Some(count_results::Variant::AggregateCount(total)), + })), + })), + metadata: Some( + self.response_metadata_v0(platform_state, CheckpointUsed::Current), + ), + } + } else { + GetDocumentsResponseV1 { + result: Some(get_documents_response_v1::Result::Data(ResultData { + variant: Some(result_data::Variant::Counts(CountResults { + variant: Some(count_results::Variant::Entries(CountEntries { + entries: entries.into_iter().map(into_v1_entry).collect(), + })), + })), + })), + metadata: Some( + self.response_metadata_v0(platform_state, CheckpointUsed::Current), + ), + } + } + } + DocumentCountResponse::Proof(proof_bytes) => { + let (grovedb_used, proof) = + self.response_proof_v0(platform_state, proof_bytes, GroveDBToUse::Current)?; + GetDocumentsResponseV1 { + result: Some(get_documents_response_v1::Result::Proof(proof)), + metadata: Some(self.response_metadata_v0(platform_state, grovedb_used)), + } + } + }; + + Ok(QueryValidationResult::new_with_data(response)) + } +} + +fn into_v1_entry(e: SplitCountEntry) -> CountEntry { + CountEntry { + in_key: e.in_key, + key: e.key, + // The wire `count` is `uint64`, so it can only carry + // `Some(_)`. Server-side never emits `None` entries to + // begin with — `None` is the SDK-side synthesis signal for + // "caller's In array contained a value the proof was + // silent on," and that decision lives client-side because + // the wire never has the caller's full In array context. + // `unwrap_or(0)` is defense-in-depth: a future executor + // bug emitting `None` shouldn't crash the response path, + // it should round to zero on the wire (matching the + // proto's `uint64` default). + count: e.count.unwrap_or(0), + } +} diff --git a/packages/rs-drive-abci/src/query/document_query/v1/dispatch/documents.rs b/packages/rs-drive-abci/src/query/document_query/v1/dispatch/documents.rs new file mode 100644 index 00000000000..9dd9419363c --- /dev/null +++ b/packages/rs-drive-abci/src/query/document_query/v1/dispatch/documents.rs @@ -0,0 +1,83 @@ +//! `RoutingDecision::Documents` — the matched-documents fetch path. + +use crate::error::Error; +use crate::platform_types::platform::Platform; +use crate::platform_types::platform_state::PlatformState; +use crate::query::QueryValidationResult; +use dapi_grpc::platform::v0::get_documents_request::get_documents_request_v0::Start as RequestV0Start; +use dapi_grpc::platform::v0::get_documents_request::get_documents_request_v1::Start as RequestV1Start; +use dapi_grpc::platform::v0::get_documents_response::get_documents_response_v1::{ + result_data, Documents, ResultData, +}; +use dapi_grpc::platform::v0::get_documents_response::{ + get_documents_response_v0, get_documents_response_v1, GetDocumentsResponseV1, +}; +use dpp::version::PlatformVersion; +use drive::query::{OrderClause, WhereClause}; + +impl Platform { + /// Forward a `select = DOCUMENTS` request through the shared + /// `query_documents_typed` helper that v0 also dispatches into. + /// v1 doesn't add any documents-side capability — the SQL-shaped + /// fields (`select`, `group_by`, `having`) are all validated as + /// documents-compatible above (empty `group_by`, empty `having`, + /// etc.) before reaching here. + #[allow(clippy::too_many_arguments)] + pub(in crate::query::document_query::v1) fn dispatch_documents_v1( + &self, + data_contract_id: Vec, + document_type: String, + where_clauses: Vec, + order_by_clauses: Vec, + limit: Option, + start: Option, + prove: bool, + platform_state: &PlatformState, + platform_version: &PlatformVersion, + ) -> Result, Error> { + let start = start.map(|s| match s { + RequestV1Start::StartAfter(b) => RequestV0Start::StartAfter(b), + RequestV1Start::StartAt(b) => RequestV0Start::StartAt(b), + }); + // `limit` is `optional uint32` on v1; the typed helper takes + // `Option` directly (`None` → server default). `Some(0)` + // can't reach here — `validate_and_route` rejects it for + // every SELECT mode so the v1 contract is uniform; only + // `None` or `Some(N > 0)` survive. + let result = self.query_documents_typed( + data_contract_id, + document_type, + where_clauses, + order_by_clauses, + limit, + prove, + start, + platform_state, + platform_version, + )?; + Ok(result.map(translate_documents_v0_to_v1)) + } +} + +/// Translate a v0 `GetDocumentsResponseV0` into v1's response +/// envelope (Documents-or-Proof wrapping the v0 oneof result into +/// v1's `ResultData`-or-`Proof` shape). +fn translate_documents_v0_to_v1( + response_v0: dapi_grpc::platform::v0::get_documents_response::GetDocumentsResponseV0, +) -> GetDocumentsResponseV1 { + let metadata = response_v0.metadata; + let result = match response_v0.result { + Some(get_documents_response_v0::Result::Documents(docs)) => { + Some(get_documents_response_v1::Result::Data(ResultData { + variant: Some(result_data::Variant::Documents(Documents { + documents: docs.documents, + })), + })) + } + Some(get_documents_response_v0::Result::Proof(proof)) => { + Some(get_documents_response_v1::Result::Proof(proof)) + } + None => None, + }; + GetDocumentsResponseV1 { result, metadata } +} diff --git a/packages/rs-drive-abci/src/query/document_query/v1/dispatch/having.rs b/packages/rs-drive-abci/src/query/document_query/v1/dispatch/having.rs new file mode 100644 index 00000000000..eb561a62775 --- /dev/null +++ b/packages/rs-drive-abci/src/query/document_query/v1/dispatch/having.rs @@ -0,0 +1,162 @@ +//! `RoutingDecision::HavingRange` — the boolean-HAVING range +//! surface (PV14). + +use super::{empty_ranking_proof_rejection, into_v1_ranked_entry}; +use crate::error::query::QueryError; +use crate::error::Error; +use crate::platform_types::platform::Platform; +use crate::platform_types::platform_state::PlatformState; +use crate::query::response_metadata::CheckpointUsed; +use crate::query::QueryValidationResult; +use dapi_grpc::platform::v0::get_documents_request::get_documents_request_v1::Start as RequestV1Start; +use dapi_grpc::platform::v0::get_documents_response::get_documents_response_v1::{ + result_data, RankedEntries, ResultData, +}; +use dapi_grpc::platform::v0::get_documents_response::{ + get_documents_response_v1, GetDocumentsResponseV1, +}; +use dpp::check_validation_result_with_data; +use dpp::data_contract::accessors::v0::DataContractV0Getters; +use dpp::identifier::Identifier; +use dpp::validation::ValidationResult; +use dpp::version::PlatformVersion; +use drive::error::query::QuerySyntaxError; +use drive::query::{ + DocumentHavingRequest, DocumentHavingResponse, HavingClause, OrderClause, SelectProjection, + WhereClause, +}; +use drive::util::grove_operations::GroveDBToUse; + +impl Platform { + /// Dispatch a boolean-`HAVING` range request + /// (`GROUP BY p HAVING LIMIT n`) to + /// [`Drive::execute_document_having_request`] and map the response + /// onto the wire. + /// + /// Parallels [`Self::dispatch_ranked_v1`] line-for-line — same + /// contract/doctype resolution, same error → typed-rejection + /// mapping, same prove split — because the two surfaces read the + /// same indexed tree. The response reuses the `RankedEntries` + /// message (a having page is the same "group key + aggregate value" + /// entry list), with one deliberate difference: `skipped` is left + /// unset. Its published contract is "the page's starting rank", and + /// a value-bounded page has no rank base — the entries are simply + /// every matching group in axis order, cut at `limit`. + #[allow(clippy::too_many_arguments)] + pub(in crate::query::document_query::v1) fn dispatch_having_v1( + &self, + data_contract_id: Vec, + document_type_name: String, + select: SelectProjection, + group_by: Vec, + having: Vec, + where_clauses: Vec, + order_clauses: Vec, + limit: Option, + offset: Option, + start: Option, + prove: bool, + platform_state: &PlatformState, + platform_version: &PlatformVersion, + ) -> Result, Error> { + let contract_id: Identifier = + check_validation_result_with_data!(data_contract_id.try_into().map_err(|_| { + QueryError::InvalidArgument( + "id must be a valid identifier (32 bytes long)".to_string(), + ) + })); + + let (_, contract_fetch_info) = self.drive.get_contract_with_fetch_info_and_fee( + contract_id.to_buffer(), + None, + true, + None, + platform_version, + )?; + let contract_fetch_info = check_validation_result_with_data!(contract_fetch_info.ok_or( + QueryError::Query(QuerySyntaxError::DataContractNotFound( + "contract not found when querying from value with contract info", + )) + )); + let contract_ref = &contract_fetch_info.contract; + let document_type = check_validation_result_with_data!(contract_ref + .document_type_for_name(document_type_name.as_str()) + .map_err(|_| QueryError::InvalidArgument(format!( + "document type {} not found for contract {}", + document_type_name, contract_id + )))); + + let drive_request = DocumentHavingRequest { + contract: contract_ref, + document_type, + group_by: &group_by, + select, + having: &having, + order_by: &order_clauses, + where_clauses: &where_clauses, + limit, + offset, + has_start_at: start.is_some(), + prove, + }; + + let drive_response = + match self + .drive + .execute_document_having_request(drive_request, None, platform_version) + { + Ok(r) => r, + Err(drive::error::Error::Query(qe)) => { + return Ok(QueryValidationResult::new_with_error(QueryError::Query(qe))); + } + // Same empty-tree mapping as the ranked path — and + // genuinely reachable here: the range prover has no + // empty-range shape for a completely empty axis + // secondary (pinned by + // `an_empty_match_set_reads_empty_and_proves_empty` in + // rs-drive's having suite), so a proved HAVING request + // against an index with no documents yet surfaces this + // merk-level failure. + Err(e) => match empty_ranking_proof_rejection(&e) { + Some(rejection) => { + return Ok(QueryValidationResult::new_with_error(rejection)); + } + None => return Err(e.into()), + }, + }; + + let response = match drive_response { + DocumentHavingResponse::Entries(entries) => GetDocumentsResponseV1 { + result: Some(get_documents_response_v1::Result::Data(ResultData { + // Always a list, never an aggregate collapse — same + // rationale as ranked: even one matching group is + // an entry, because the caller needs to know which + // group matched, not only that one did. + variant: Some(result_data::Variant::Ranked(RankedEntries { + // Order preserved verbatim: axis order in the + // walk direction, and drive already asserted + // the list is no longer than the limit. + entries: entries.into_iter().map(into_v1_ranked_entry).collect(), + // Deliberately unset. `skipped`'s published + // contract is rank-based ("entry i is the + // group at rank skipped + i"), and a + // value-bounded page has no rank base — there + // is nothing the field could truthfully say. + skipped: None, + })), + })), + metadata: Some(self.response_metadata_v0(platform_state, CheckpointUsed::Current)), + }, + DocumentHavingResponse::Proof(proof_bytes) => { + let (grovedb_used, proof) = + self.response_proof_v0(platform_state, proof_bytes, GroveDBToUse::Current)?; + GetDocumentsResponseV1 { + result: Some(get_documents_response_v1::Result::Proof(proof)), + metadata: Some(self.response_metadata_v0(platform_state, grovedb_used)), + } + } + }; + + Ok(QueryValidationResult::new_with_data(response)) + } +} diff --git a/packages/rs-drive-abci/src/query/document_query/v1/dispatch/mod.rs b/packages/rs-drive-abci/src/query/document_query/v1/dispatch/mod.rs new file mode 100644 index 00000000000..4c2d0315f48 --- /dev/null +++ b/packages/rs-drive-abci/src/query/document_query/v1/dispatch/mod.rs @@ -0,0 +1,107 @@ +//! Per-route executors for the v1 `getDocuments` handler — one file +//! per `RoutingDecision` arm, split from the handler's `mod.rs` for +//! readability. Each file holds one `impl Platform` block with +//! that route's dispatcher; helpers shared by exactly one dispatcher +//! live next to it, and helpers shared by the ranked + having pair +//! (which reuse the same wire entry shape) live here. + +mod average; +mod count; +mod documents; +mod having; +mod ranked; +mod sum; + +use crate::error::query::QueryError; +use dapi_grpc::platform::v0::get_documents_response::get_documents_response_v1::{ + ranked_entry, RankedEntry, +}; +use drive::query::{RankedEntry as DriveRankedEntry, RankedEntryValue}; + +/// Translate an rs-drive `RankedEntry` into the wire `RankedEntry`. +/// Mirror of [`into_v1_entry`] / [`into_v1_sum_entry`] / +/// [`into_v1_average_entry`] for the ranked surface. +/// +/// `key` passes through as the raw index-key bytes of the grouped +/// property value — the same bytes the proof commits to, so a client +/// verifying the proof compares byte-for-byte against what it +/// reconstructs. +/// +/// The `value` oneof is always set: drive's `RankedEntryValue` has no +/// "absent" variant (unlike the count / sum entry types, whose +/// `Option` exists for the SDK's synthesize-for-missing-In-value +/// concept — a ranked result has no caller-supplied key set to be +/// silent about). +fn into_v1_ranked_entry(e: DriveRankedEntry) -> RankedEntry { + RankedEntry { + key: e.key, + value: Some(match e.value { + RankedEntryValue::Count(count) => ranked_entry::Value::Count(count), + RankedEntryValue::Sum(sum) => ranked_entry::Value::Sum(sum), + // The wire carries a `double` approximation of the exact + // fixed-point `i128` the Avg axis is ordered by: + // `fixed_point as f64 / RANKED_AVG_SCALE as f64`, which is + // what `as_f64` computes. Lossy by construction, and that + // is fine — these entries are only read on the no-proof + // ("quick answer") path. A proof-verifying client ignores + // this field and reconstructs the exact fixed point from + // the grovedb proof, so no verification depends on it. + // Ranking order is still exact: the ordering happened over + // the i128 before this conversion. + value @ RankedEntryValue::AvgFixedPoint(_) => ranked_entry::Value::Avg(value.as_f64()), + }), + } +} + +/// Recognize the one grovedb failure that is a caller-facing +/// condition rather than a server fault: **an empty ranking cannot be +/// proved**. +/// +/// **This is now a backstop rather than a live path.** The ranked +/// prover moved to `prove_indexed_axis_top_k_paginated`, which emits a +/// guaranteed-empty range against an empty axis secondary instead of +/// refusing, so proving a ranking over a contract with no documents +/// succeeds and the proved and unproven paths agree (pinned by +/// `ranked_tests::proving_an_empty_ranking_succeeds`). The mapping is +/// kept because the failure it recognizes is a *class* — a merk-level +/// "cannot prove an empty tree" surfacing from somewhere in the +/// ancestor chain — not a single call site, and because the cost of +/// keeping it is one string comparison on an error path. +/// +/// Historically: the non-paginated prover had no absence-proof shape +/// for "this axis secondary has no entries", so proving a ranking over +/// an index that held no documents failed with a merk-level "Cannot +/// create proof for empty tree", wrapped by grovedb as +/// `CorruptedData`. Reaching that state needed nothing exotic — +/// querying a freshly registered contract with `prove = true` did it — +/// so letting it propagate would answer an ordinary request with an +/// internal error (`Status::unknown`) and an alarming server-side log +/// line, and give the caller no idea that the same request without +/// `prove` succeeded and returned the empty list. +/// +/// Detection is by variant + marker substring rather than by a typed +/// error, because grovedb flattens the merk error into a +/// `CorruptedData(String)` at the indexed-axis proof boundary; the +/// substring is the merk-side constant. The match is deliberately +/// narrow: any other `CorruptedData` still propagates as an internal +/// error, because for every other cause that classification is +/// correct. +fn empty_ranking_proof_rejection(error: &drive::error::Error) -> Option { + let drive::error::Error::GroveDB(grove_error) = error else { + return None; + }; + let drive::query::GroveError::CorruptedData(message) = grove_error.as_ref() else { + return None; + }; + if !message.contains("Cannot create proof for empty tree") { + return None; + } + Some(QueryError::InvalidArgument( + "this index's axis secondary has no groups yet, and an empty ranking or \ + HAVING range cannot be proved: grovedb has no absence-proof shape for \ + an empty axis secondary. Retry with `prove = false` — the unproven \ + read answers the same request with an empty entry list. Once the \ + index holds at least one document, the proved form works." + .to_string(), + )) +} diff --git a/packages/rs-drive-abci/src/query/document_query/v1/dispatch/ranked.rs b/packages/rs-drive-abci/src/query/document_query/v1/dispatch/ranked.rs new file mode 100644 index 00000000000..2d75bc4b4f1 --- /dev/null +++ b/packages/rs-drive-abci/src/query/document_query/v1/dispatch/ranked.rs @@ -0,0 +1,174 @@ +//! `RoutingDecision::Ranked` — the ranked top-k surface (PV14). + +use super::{empty_ranking_proof_rejection, into_v1_ranked_entry}; +use crate::error::query::QueryError; +use crate::error::Error; +use crate::platform_types::platform::Platform; +use crate::platform_types::platform_state::PlatformState; +use crate::query::response_metadata::CheckpointUsed; +use crate::query::QueryValidationResult; +use dapi_grpc::platform::v0::get_documents_request::get_documents_request_v1::Start as RequestV1Start; +use dapi_grpc::platform::v0::get_documents_response::get_documents_response_v1::{ + result_data, RankedEntries, ResultData, +}; +use dapi_grpc::platform::v0::get_documents_response::{ + get_documents_response_v1, GetDocumentsResponseV1, +}; +use dpp::check_validation_result_with_data; +use dpp::data_contract::accessors::v0::DataContractV0Getters; +use dpp::identifier::Identifier; +use dpp::validation::ValidationResult; +use dpp::version::PlatformVersion; +use drive::error::query::QuerySyntaxError; +use drive::query::{ + DocumentRankedRequest, DocumentRankedResponse, HavingClause, OrderClause, SelectProjection, + WhereClause, +}; +use drive::util::grove_operations::GroveDBToUse; + +impl Platform { + /// Dispatch a ranked request — a `COUNT` / `SUM` / `AVG` select + /// with a `GROUP BY` whose single `ORDER BY` clause names the + /// selected aggregate — to + /// [`Drive::execute_document_ranked_request`], and map the response + /// into a `GetDocumentsResponseV1` carrying a `RankedEntries` + /// payload (or a `Proof` payload when prove=true). + /// + /// Structurally parallel to [`Self::dispatch_count_v1`] — same + /// contract fetch, same `Error::Query` → typed-rejection mapping, + /// same prove vs no-prove split — with two differences worth + /// naming: + /// + /// 1. **The request is forwarded whole.** `where_clauses`, + /// `having`, `order_by`, `limit`, `offset` and `start` all go + /// down, including the ones a ranked request must leave empty, + /// because drive owns those rejections: the SDK's client-side + /// helpers call drive's validator with no abci in the path, so + /// re-checking here would create a second, driftable copy of + /// the grammar. The rejections come back as `Error::Query(...)` + /// and are surfaced to the caller as query errors, not internal + /// ones. `order_by` in particular is no longer refused here — + /// it is the ranking. + /// 2. **Proving an empty ranking is mapped, not propagated.** See + /// [`empty_ranking_proof_rejection`]. + #[allow(clippy::too_many_arguments)] + pub(in crate::query::document_query::v1) fn dispatch_ranked_v1( + &self, + data_contract_id: Vec, + document_type_name: String, + select: SelectProjection, + group_by: Vec, + having: Vec, + where_clauses: Vec, + order_clauses: Vec, + limit: Option, + offset: Option, + start: Option, + prove: bool, + platform_state: &PlatformState, + platform_version: &PlatformVersion, + ) -> Result, Error> { + let contract_id: Identifier = + check_validation_result_with_data!(data_contract_id.try_into().map_err(|_| { + QueryError::InvalidArgument( + "id must be a valid identifier (32 bytes long)".to_string(), + ) + })); + + let (_, contract_fetch_info) = self.drive.get_contract_with_fetch_info_and_fee( + contract_id.to_buffer(), + None, + true, + None, + platform_version, + )?; + let contract_fetch_info = check_validation_result_with_data!(contract_fetch_info.ok_or( + QueryError::Query(QuerySyntaxError::DataContractNotFound( + "contract not found when querying from value with contract info", + )) + )); + let contract_ref = &contract_fetch_info.contract; + let document_type = check_validation_result_with_data!(contract_ref + .document_type_for_name(document_type_name.as_str()) + .map_err(|_| QueryError::InvalidArgument(format!( + "document type {} not found for contract {}", + document_type_name, contract_id + )))); + + let drive_request = DocumentRankedRequest { + contract: contract_ref, + document_type, + group_by: &group_by, + select, + having: &having, + order_by: &order_clauses, + where_clauses: &where_clauses, + limit, + offset, + has_start_at: start.is_some(), + prove, + }; + + let drive_response = + match self + .drive + .execute_document_ranked_request(drive_request, None, platform_version) + { + Ok(r) => r, + Err(drive::error::Error::Query(qe)) => { + return Ok(QueryValidationResult::new_with_error(QueryError::Query(qe))); + } + Err(e) => match empty_ranking_proof_rejection(&e) { + Some(rejection) => { + return Ok(QueryValidationResult::new_with_error(rejection)); + } + None => return Err(e.into()), + }, + }; + + let response = match drive_response { + DocumentRankedResponse::Entries(page) => GetDocumentsResponseV1 { + result: Some(get_documents_response_v1::Result::Data(ResultData { + // No aggregate-collapse arm here, unlike count / + // sum / average: a ranked result is always a list + // of groups. Even `LIMIT 1` returns one *entry*, + // because the caller needs to know which group + // won, not only the winning value. + variant: Some(result_data::Variant::Ranked(RankedEntries { + // Order is preserved verbatim: entry order is + // the ranking order, and drive already + // asserted the list is no longer than `k`. + entries: page.entries.into_iter().map(into_v1_ranked_entry).collect(), + // The page's starting rank, so entry `i` is + // identifiable as the group at rank + // `skipped + i` rather than as "one of the + // top few". This is the *unproven* path, so + // the number is only as good as the node — + // which is exactly why a proving client + // ignores it and re-derives the attested + // value from the proof bytes instead (see + // `RankedPage::skipped`). Sent as `Some` + // unconditionally, including the `0` an + // offset-less query produces: the proto field + // is `optional` to keep "this node predates + // the field" distinguishable from "this page + // starts at rank 0", and collapsing 0 to + // `None` would throw that distinction away. + skipped: Some(page.skipped), + })), + })), + metadata: Some(self.response_metadata_v0(platform_state, CheckpointUsed::Current)), + }, + DocumentRankedResponse::Proof(proof_bytes) => { + let (grovedb_used, proof) = + self.response_proof_v0(platform_state, proof_bytes, GroveDBToUse::Current)?; + GetDocumentsResponseV1 { + result: Some(get_documents_response_v1::Result::Proof(proof)), + metadata: Some(self.response_metadata_v0(platform_state, grovedb_used)), + } + } + }; + + Ok(QueryValidationResult::new_with_data(response)) + } +} diff --git a/packages/rs-drive-abci/src/query/document_query/v1/dispatch/sum.rs b/packages/rs-drive-abci/src/query/document_query/v1/dispatch/sum.rs new file mode 100644 index 00000000000..5a495cef98d --- /dev/null +++ b/packages/rs-drive-abci/src/query/document_query/v1/dispatch/sum.rs @@ -0,0 +1,214 @@ +//! `RoutingDecision::Sum` — the grouped sum surface. + +use super::super::not_yet_implemented; +use crate::error::query::QueryError; +use crate::error::Error; +use crate::platform_types::platform::Platform; +use crate::platform_types::platform_state::PlatformState; +use crate::query::response_metadata::CheckpointUsed; +use crate::query::QueryValidationResult; +use dapi_grpc::platform::v0::get_documents_request::get_documents_request_v1::Start as RequestV1Start; +use dapi_grpc::platform::v0::get_documents_response::get_documents_response_v1::{ + result_data, sum_results, ResultData, SumEntries, SumEntry, SumResults, +}; +use dapi_grpc::platform::v0::get_documents_response::{ + get_documents_response_v1, GetDocumentsResponseV1, +}; +use dpp::check_validation_result_with_data; +use dpp::data_contract::accessors::v0::DataContractV0Getters; +use dpp::identifier::Identifier; +use dpp::validation::ValidationResult; +use dpp::version::PlatformVersion; +use drive::error::query::QuerySyntaxError; +use drive::query::{ + CountMode, DocumentSumRequest, DocumentSumResponse, OrderClause, SumEntry as DriveSumEntry, + SumMode, WhereClause, +}; +use drive::util::grove_operations::GroveDBToUse; + +impl Platform { + /// Dispatch a `select = SUM(field)` request to + /// [`Drive::execute_document_sum_request`] and map the response + /// into a `GetDocumentsResponseV1` carrying a `SumResults` payload + /// (or a `Proof` payload when prove=true). + /// + /// Parallels [`Self::dispatch_count_v1`] line-by-line — same + /// request construction, same error → typed-rejection mapping, + /// same prove vs no-prove split. Only the response shape mapping + /// differs: `DocumentSumResponse::Aggregate(i64)` → + /// `SumResults::aggregate_sum`, `Entries(Vec)` → + /// `SumResults::entries`, `Proof(bytes)` → outer `result.proof`. + #[allow(clippy::too_many_arguments)] + pub(in crate::query::document_query::v1) fn dispatch_sum_v1( + &self, + data_contract_id: Vec, + document_type_name: String, + where_clauses: Vec, + order_clauses: Vec, + limit: Option, + start: Option, + prove: bool, + sum_property: String, + mode: CountMode, + platform_state: &PlatformState, + platform_version: &PlatformVersion, + ) -> Result, Error> { + if start.is_some() { + return Ok(QueryValidationResult::new_with_error(not_yet_implemented( + "start_after / start_at with SELECT SUM (paginate by narrowing the \ + range clause itself)", + ))); + } + + let contract_id: Identifier = + check_validation_result_with_data!(data_contract_id.try_into().map_err(|_| { + QueryError::InvalidArgument( + "id must be a valid identifier (32 bytes long)".to_string(), + ) + })); + + let (_, contract_fetch_info) = self.drive.get_contract_with_fetch_info_and_fee( + contract_id.to_buffer(), + None, + true, + None, + platform_version, + )?; + let contract_fetch_info = check_validation_result_with_data!(contract_fetch_info.ok_or( + QueryError::Query(QuerySyntaxError::DataContractNotFound( + "contract not found when querying from value with contract info", + )) + )); + let contract_ref = &contract_fetch_info.contract; + let document_type = check_validation_result_with_data!(contract_ref + .document_type_for_name(document_type_name.as_str()) + .map_err(|_| QueryError::InvalidArgument(format!( + "document type {} not found for contract {}", + document_type_name, contract_id + )))); + + // `SumMode` mirrors `CountMode` 1:1 — same four variants + // computed via the same `compute_aggregate_mode_and_check_limit` + // helper. Map across the isomorphism. + let sum_mode = match mode { + CountMode::Aggregate => SumMode::Aggregate, + CountMode::GroupByIn => SumMode::GroupByIn, + CountMode::GroupByRange => SumMode::GroupByRange, + CountMode::GroupByCompound => SumMode::GroupByCompound, + }; + + let drive_request = DocumentSumRequest { + contract: contract_ref, + document_type, + sum_property, + where_clauses, + order_clauses, + mode: sum_mode, + limit, + prove, + drive_config: &self.config.drive, + }; + let drive_response = + match self + .drive + .execute_document_sum_request(drive_request, None, platform_version) + { + Ok(r) => r, + Err(drive::error::Error::Query(qe)) => { + return Ok(QueryValidationResult::new_with_error(QueryError::Query(qe))); + } + Err(e) => return Err(e.into()), + }; + + let response = match drive_response { + DocumentSumResponse::Aggregate(sum) => GetDocumentsResponseV1 { + result: Some(get_documents_response_v1::Result::Data(ResultData { + variant: Some(result_data::Variant::Sums(SumResults { + variant: Some(sum_results::Variant::AggregateSum(sum)), + })), + })), + metadata: Some(self.response_metadata_v0(platform_state, CheckpointUsed::Current)), + }, + DocumentSumResponse::Entries(entries) => { + if sum_mode == SumMode::Aggregate { + // Mirror of count's same-arm: `select=SUM, + // group_by=[]` whose executor routed through a + // PerInValue path (In + no range + no prove) + // returns one entry per In branch. Fold them into + // a single aggregate. `checked_add` surfaces the + // narrow case where per-branch sums truly add to + // more than i64::MAX as a typed + // `QuerySyntaxError::Unsupported` rather than + // silently saturating at i64::MAX (which produces + // a deterministic-but-misleading answer). + let mut total: i64 = 0; + let mut overflow = false; + for e in &entries { + match total.checked_add(e.sum.unwrap_or(0)) { + Some(t) => total = t, + None => { + overflow = true; + break; + } + } + } + if overflow { + return Ok(QueryValidationResult::new_with_error(QueryError::Query( + QuerySyntaxError::Unsupported( + "aggregate SUM across In branches overflows i64 — \ + the In-fold cannot be represented; narrow the In set \ + or query branches individually" + .to_string(), + ), + ))); + } + GetDocumentsResponseV1 { + result: Some(get_documents_response_v1::Result::Data(ResultData { + variant: Some(result_data::Variant::Sums(SumResults { + variant: Some(sum_results::Variant::AggregateSum(total)), + })), + })), + metadata: Some( + self.response_metadata_v0(platform_state, CheckpointUsed::Current), + ), + } + } else { + GetDocumentsResponseV1 { + result: Some(get_documents_response_v1::Result::Data(ResultData { + variant: Some(result_data::Variant::Sums(SumResults { + variant: Some(sum_results::Variant::Entries(SumEntries { + entries: entries.into_iter().map(into_v1_sum_entry).collect(), + })), + })), + })), + metadata: Some( + self.response_metadata_v0(platform_state, CheckpointUsed::Current), + ), + } + } + } + DocumentSumResponse::Proof(proof_bytes) => { + let (grovedb_used, proof) = + self.response_proof_v0(platform_state, proof_bytes, GroveDBToUse::Current)?; + GetDocumentsResponseV1 { + result: Some(get_documents_response_v1::Result::Proof(proof)), + metadata: Some(self.response_metadata_v0(platform_state, grovedb_used)), + } + } + }; + + Ok(QueryValidationResult::new_with_data(response)) + } +} + +/// Translate an rs-drive `SumEntry` into the wire `SumEntry`. Mirror +/// of [`into_v1_entry`] for the sum surface. +fn into_v1_sum_entry(e: DriveSumEntry) -> SumEntry { + SumEntry { + in_key: e.in_key, + key: e.key, + // `sum` is `sint64` on the wire — same `None`-rounds-to-0 + // contract as `into_v1_entry`. + sum: e.sum.unwrap_or(0), + } +} diff --git a/packages/rs-drive-abci/src/query/document_query/v1/mod.rs b/packages/rs-drive-abci/src/query/document_query/v1/mod.rs index def61a80a91..4a7fe596e1c 100644 --- a/packages/rs-drive-abci/src/query/document_query/v1/mod.rs +++ b/packages/rs-drive-abci/src/query/document_query/v1/mod.rs @@ -28,42 +28,39 @@ mod compute_aggregate_mode_and_check_limit; mod conversions; +mod dispatch; +mod routing; -use self::compute_aggregate_mode_and_check_limit::{ - compute_aggregate_mode_and_check_limit, AggregateRouting, +use routing::{reject_offset_off_the_ranked_path, validate_and_route}; +// Re-exported so `tests.rs` (a `use super::*` consumer) keeps seeing +// the routing probe under its old name. +#[cfg(test)] +use routing::validate_and_route_for_tests; + +// Names below are consumed only by `tests.rs`, which imports this +// module's scope wholesale via `use super::*`. +#[cfg(test)] +use { + dapi_grpc::platform::v0::get_documents_response::get_documents_response_v0, + dapi_grpc::platform::v0::get_documents_response::get_documents_response_v1::{ + self, count_results, ranked_entry, result_data, CountEntry, CountResults, RankedEntries, + RankedEntry, ResultData, + }, + dpp::data_contract::accessors::v0::DataContractV0Getters, + dpp::identifier::Identifier, + drive::query::WhereClause, }; use crate::error::query::QueryError; use crate::error::Error; use crate::platform_types::platform::Platform; use crate::platform_types::platform_state::PlatformState; -use crate::query::response_metadata::CheckpointUsed; use crate::query::QueryValidationResult; -use dapi_grpc::platform::v0::get_documents_request::get_documents_request_v0::Start as RequestV0Start; -use dapi_grpc::platform::v0::get_documents_request::get_documents_request_v1::Start as RequestV1Start; use dapi_grpc::platform::v0::get_documents_request::GetDocumentsRequestV1; -use dapi_grpc::platform::v0::get_documents_response::get_documents_response_v1::{ - average_results, count_results, ranked_entry, result_data, sum_results, AverageAggregate, - AverageEntries, AverageEntry, AverageResults, CountEntries, CountEntry, CountResults, - Documents, RankedEntries, RankedEntry, ResultData, SumEntries, SumEntry, SumResults, -}; -use dapi_grpc::platform::v0::get_documents_response::{ - get_documents_response_v0, get_documents_response_v1, GetDocumentsResponseV1, -}; -use dpp::check_validation_result_with_data; -use dpp::data_contract::accessors::v0::DataContractV0Getters; -use dpp::identifier::Identifier; -use dpp::validation::ValidationResult; +use dapi_grpc::platform::v0::get_documents_response::GetDocumentsResponseV1; use dpp::version::PlatformVersion; use drive::error::query::QuerySyntaxError; -use drive::query::{ - AverageEntry as DriveAverageEntry, AverageMode, CountMode, DocumentAverageRequest, - DocumentAverageResponse, DocumentCountRequest, DocumentCountResponse, DocumentRankedRequest, - DocumentRankedResponse, DocumentSumRequest, DocumentSumResponse, HavingClause, OrderClause, - RankedEntry as DriveRankedEntry, RankedEntryValue, SelectFunction, SelectProjection, - SplitCountEntry, SumEntry as DriveSumEntry, SumMode, WhereClause, -}; -use drive::util::grove_operations::GroveDBToUse; +use drive::query::{CountMode, SelectProjection}; /// Build a `QuerySyntaxError::Unsupported` carrying a stable /// " is not yet implemented" message. The wording is @@ -78,259 +75,6 @@ pub(super) fn not_yet_implemented(feature: &str) -> QueryError { ))) } -/// Validate the `select` × `group_by` × `order_by` × `having` -/// combination against the supported-shape table (see the -/// message-level docstring on `GetDocumentsRequestV1` in -/// `platform.proto`). Returns the routing decision so the handler -/// knows whether to dispatch to the documents-fetch path, the count -/// path or the ranked path, and which response shape to produce. -#[allow(clippy::too_many_arguments)] -fn validate_and_route( - select: &SelectProjection, - limit: Option, - having: &[HavingClause], - group_by: &[String], - order_by: &[OrderClause], - where_clauses: &[WhereClause], - platform_version: &PlatformVersion, -) -> Result { - // Centralized `limit: Some(0)` rejection. - // - // `limit` is `optional uint32` on the wire, so `Some(0)` is a - // distinct value any raw-gRPC/WASM/FFI caller can encode. Three - // legacy behaviors collide on this value across the v1 dispatch - // surface: - // - `SELECT DOCUMENTS` would `unwrap_or(0)` and forward to v0, - // where `limit=0` is the v0-uint32 sentinel for "use server - // default" — accept-as-default. - // - `SELECT COUNT` with `mode ∈ {Aggregate, GroupByIn}` would - // reject via the `is_some()` check below — reject-as-invalid. - // - `SELECT COUNT` with `mode ∈ {GroupByRange, GroupByCompound}` - // would pass `Some(0)` through to drive, which honors it as a - // zero-cap walk — accept-as-zero. - // - // Three semantics for the same wire bytes is bad contract. The - // v1 wire's whole point of switching to `optional uint32` was - // to make "unset" explicit (`None`), so `Some(0)` only makes - // sense as an *explicit* zero — and a zero-cap query returns - // no useful information regardless of mode. Reject it uniformly - // at the validation boundary so callers see a single, - // mode-independent contract: `None` for "use server default", - // `Some(N > 0)` for an explicit cap, `Some(0)` is invalid. - if limit == Some(0) { - return Err(QueryError::Query(QuerySyntaxError::InvalidLimit( - "limit = 0 is not a valid wire value on the v1 \ - `optional uint32` field; omit `limit` (None) to use the \ - server's default, or pass a positive integer for an \ - explicit cap (a zero-cap query is structurally \ - meaningless regardless of SELECT mode)" - .to_string(), - ))); - } - - // HAVING is only ever meaningful for an aggregate projection: - // it is a boolean predicate over the groups a `COUNT` / `SUM` / - // `AVG` produces. Those three functions route through the - // versioned `compute_aggregate_mode_and_check_limit` helper below, - // which rejects non-empty HAVING on both tables (evaluation is not - // implemented) but with wording that depends on whether the - // request is otherwise a ranked one. - // - // For every other SELECT there is no aggregate for a HAVING to - // talk about, so the rejection stays here and stays unversioned. - // It also stays *ahead* of the per-function gates below, exactly - // where the old blanket rejection was: `SELECT DOCUMENTS … HAVING` - // must keep reporting the HAVING rather than silently dropping it - // on the documents path, and `SELECT MIN/MAX … HAVING` must not - // start reporting MIN/MAX as the reason a request with two - // unsupported features was refused. - if !having.is_empty() - && !matches!( - select.function, - SelectFunction::Count | SelectFunction::Sum | SelectFunction::Avg - ) - { - return Err(not_yet_implemented("HAVING clause")); - } - - match select.function { - SelectFunction::Documents => { - if !select.field.is_empty() { - return Err(QueryError::InvalidArgument(format!( - "SELECT DOCUMENTS does not accept a projection field; \ - got field='{}' (omit the field for plain document fetch, \ - or use SELECT COUNT / SUM / AVG to project a value)", - select.field - ))); - } - if !group_by.is_empty() { - // GROUP BY with SELECT DOCUMENTS is structurally - // nonsensical — GROUP BY produces one row per - // distinct key, but SELECT DOCUMENTS returns the - // underlying rows; the two contracts can't be - // reconciled. Callers wanting per-group output use - // SELECT COUNT / SUM / AVG / MIN / MAX. Classify - // as `InvalidArgument` rather than - // `not_yet_implemented` because this isn't a - // future capability — no protocol version will - // make this combination meaningful. - return Err(QueryError::InvalidArgument(format!( - "GROUP BY with SELECT DOCUMENTS is not a valid SQL shape: \ - GROUP BY produces one row per distinct key, but SELECT \ - DOCUMENTS returns the underlying rows themselves. Use \ - SELECT COUNT / SUM / AVG / MIN / MAX with GROUP BY for \ - per-group output, or SELECT DOCUMENTS without GROUP BY \ - for plain document fetch. Got group_by={:?}.", - group_by - ))); - } - Ok(RoutingDecision::Documents) - } - SelectFunction::Sum => { - // SELECT SUM(field): routes to - // `Drive::execute_document_sum_request` (in - // `packages/rs-drive/src/query/drive_document_sum_query/`). - // `field` must be non-empty and must name an integer - // property on the document type that's covered by either - // `documents_summable` (doctype level) or a `summable: - // ""` index. Validation lives downstream in - // [`crate::query::drive_document_sum_query::drive_dispatcher::detect_sum_mode`]. - // - // Wiring: `RoutingDecision::Sum(...)` variant below feeds - // the dispatch arm in the response-building section, which - // routes the resulting `DocumentSumResponse` into the - // `SumResults` proto message defined in platform.proto. - if select.field.is_empty() { - return Err(QueryError::InvalidArgument( - "SELECT SUM requires a non-empty `field` naming the integer property \ - to sum (e.g. `SUM(amount)`). The contract must declare \ - `documentsSummable: \"\"` at the document-type level OR a \ - `summable: \"\"` index covering the where-clause shape; the \ - DPP validator enforces this at contract creation." - .to_string(), - )); - } - match compute_aggregate_mode_and_check_limit( - select, - group_by, - where_clauses, - order_by, - limit, - having, - "SUM", - platform_version, - )? { - AggregateRouting::Grouped(mode) => Ok(RoutingDecision::Sum { - sum_property: select.field.clone(), - mode, - }), - AggregateRouting::Ranked => Ok(RoutingDecision::Ranked), - } - } - SelectFunction::Avg => { - // SELECT AVG(field): routes to - // `Drive::execute_document_average_request` (in - // `packages/rs-drive/src/query/drive_document_average_query/`). - // `field` must be non-empty and must name an integer - // property covered by either `documents_summable` (doctype - // level) or a `summable: ""` index — averages reuse - // sum-tree indexes (no separate `averageable` flag exists - // or is needed; the same `CountSumTree` / PCPS element - // backs both). - // - // Wiring: `RoutingDecision::Average(...)` variant below - // feeds the dispatch arm in the response-building section, - // which routes the resulting `DocumentAverageResponse` into - // the `AverageResults` proto message defined in - // platform.proto. - if select.field.is_empty() { - return Err(QueryError::InvalidArgument( - "SELECT AVG requires a non-empty `field` naming the integer property \ - to average (e.g. `AVG(score)`). The contract must declare \ - `documentsSummable: \"\"` at the document-type level OR a \ - `summable: \"\"` index covering the where-clause shape; the \ - DPP validator enforces this at contract creation. Averages reuse \ - sum-tree indexes — no separate `averageable` flag is required." - .to_string(), - )); - } - match compute_aggregate_mode_and_check_limit( - select, - group_by, - where_clauses, - order_by, - limit, - having, - "AVG", - platform_version, - )? { - AggregateRouting::Grouped(mode) => Ok(RoutingDecision::Average { - sum_property: select.field.clone(), - mode, - }), - AggregateRouting::Ranked => Ok(RoutingDecision::Ranked), - } - } - SelectFunction::Min => Err(not_yet_implemented( - "SELECT MIN (the wire surface accepts MIN(field) so callers \ - can encode it ahead of server support landing, but the \ - server doesn't yet evaluate per-group MIN; semantically \ - distinct from asking for the lowest-ranked group, which is \ - `ORDER BY ASC LIMIT 1`)", - )), - SelectFunction::Max => Err(not_yet_implemented( - "SELECT MAX (the wire surface accepts MAX(field) so callers \ - can encode it ahead of server support landing, but the \ - server doesn't yet evaluate per-group MAX; semantically \ - distinct from asking for the highest-ranked group, which is \ - `ORDER BY DESC LIMIT 1`)", - )), - SelectFunction::Count => { - if !select.field.is_empty() { - return Err(not_yet_implemented( - "SELECT COUNT(field) — counting non-null values of a \ - specific field (the wire surface accepts the field so \ - callers can encode it ahead of server support landing, \ - but today only COUNT(*) — empty `field` — is evaluated)", - )); - } - // Field-membership predicates on the request's where - // clauses. **Match-any, not match-first** — a request - // may carry two range clauses on different fields - // (the executor's `RangeAggregateCarrierProof` path - // is built for exactly that shape; see - // `outer_range_plus_inner_range_with_prove_and_group_by_range_routes_to_carrier_proof` - // in `drive/query/drive_document_count_query/tests.rs`). - // A `find(...).map(field).map(eq)` test against a - // hard-coded first range clause would make the routing - // decision depend on clause ordering on the wire, - // which is wrong — `WHERE a > x AND b > y GROUP BY a` - // and `WHERE b > y AND a > x GROUP BY a` must produce - // the same routing. - // - // For `In` the practical effect is the same because - // `validate_and_canonicalize_where_clauses` rejects - // multiple `In` clauses upstream (`MultipleInClauses`), - // but the `any` shape is used here too so the routing - // logic doesn't bake in an assumption that could go - // stale if that validator's contract ever relaxes. - match compute_aggregate_mode_and_check_limit( - select, - group_by, - where_clauses, - order_by, - limit, - having, - "COUNT", - platform_version, - )? { - AggregateRouting::Grouped(mode) => Ok(RoutingDecision::Count(mode)), - AggregateRouting::Ranked => Ok(RoutingDecision::Ranked), - } - } - } -} - /// Outcome of `validate_and_route` — names the path the v1 request /// will dispatch to. /// @@ -378,140 +122,18 @@ enum RoutingDecision { /// there is nothing to carry and no opportunity for the routing /// layer's reading of the ranking to drift from drive's. Ranked, -} - -/// The `OFFSET` gate, applied **after** routing. -/// -/// Offset pagination exists on exactly one path: the ranked executor, -/// where `OFFSET m` is the rank the returned page starts at and costs -/// nothing to prove (grovedb attests the skipped region from counted -/// subtree commitments rather than walking it). Every other v1 shape — -/// documents, and the grouped count / sum / average modes — has no -/// offset primitive behind it and keeps the rejection it has always -/// had, **message for message**: those callers paginate with -/// `start_after` / `start_at`, or by narrowing the range clause. -/// -/// The message below is load-bearing and must not be reworded: clients -/// match on it, and on a protocol version whose routing table has no -/// ranked path (v13 and earlier) it is the *only* answer an offset can -/// get, exactly as it was before the ranked surface existed. -fn reject_offset_off_the_ranked_path( - offset: Option, - decision: &RoutingDecision, -) -> Result<(), QueryError> { - if offset.is_some() && !matches!(decision, RoutingDecision::Ranked) { - return Err(not_yet_implemented( - "OFFSET pagination (use cursor pagination via `start_after` / \ - `start_at` instead)", - )); - } - Ok(()) -} - -/// Test-only: expose the routing decision for unit tests without -/// needing a full `Platform` setup. Mirrors **both the rejection -/// messages and the gate ordering** of [`Platform::query_documents_v1`] -/// so a test that pins a first-fail message also pins the order -/// gates fire in, not just which gate eventually fires. -/// -/// Sequence (same as the real handler at -/// [`Platform::query_documents_v1`]): -/// 1. `where_clauses_from_proto` → propagate `InvalidArgument` / -/// `Unsupported` decode errors -/// 2. `order_clauses_from_proto` → propagate aggregate-target -/// rejection / `InvalidArgument` decode errors -/// 3. `selects.len() > 1` → `not_yet_implemented("multi-projection …")` -/// 4. `select_from_proto` (first element, or default documents) -/// 5. `having_clauses_from_proto` → propagate `InvalidArgument` -/// decode errors (unknown aggregate-function / operator -/// discriminant, missing aggregate, missing / retired right -/// operand) -/// 6. [`validate_and_route`] — which itself runs `limit == Some(0)` -/// → the non-aggregate HAVING gate → per-function gates → -/// routing pick (including the versioned ranked gate). -/// 7. `offset.is_some()` on a non-ranked decision → -/// `not_yet_implemented("OFFSET …")` -/// -/// The OFFSET gate is **last**, not first: whether an offset is -/// acceptable now depends on where the request routes (the ranked -/// executor paginates by offset; nothing else does), and that is not -/// known until routing has run. On a protocol version whose table has -/// no ranked path, every offset is still refused with the identical -/// message — only its position relative to the decode gates moved. -/// -/// Treats an unset `select` (proto-default) the same way the -/// handler does — as `SelectProjection::documents()`. -#[cfg(test)] -pub(super) fn validate_and_route_for_tests( - request_v1: &GetDocumentsRequestV1, - where_clauses: &[WhereClause], - platform_version: &PlatformVersion, -) -> Result<&'static str, QueryError> { - // 1. WHERE decoding — wire-malformed shapes (unknown operator - // discriminant, nested `DocumentFieldValue.list` beyond - // depth 1, …) reject as `InvalidArgument`. Runs even - // though the caller passes a separate pre-decoded - // `where_clauses` slice for the routing decision, because - // the depth-cap and similar decode-time contracts aren't - // exercisable otherwise. - conversions::where_clauses_from_proto(request_v1.where_clauses.clone())?; - // 2. ORDER BY decoding — aggregate-target reject as - // `Unsupported("ORDER BY on aggregate keys …")`. - let order_by_clauses = conversions::order_clauses_from_proto(request_v1.order_by.clone())?; - // 3. Multi-projection SELECT rejection. - if request_v1.selects.len() > 1 { - return Err(not_yet_implemented( - "multi-projection SELECT (the wire accepts `repeated Select` so \ - callers can encode `SELECT COUNT(*), SUM(amount), AVG(rating)` \ - ahead of server support landing, but today only single-projection \ - requests are evaluated; the response shape will gain a parallel \ - `repeated AggregateValue values` field when multi-projection \ - lands)", - )); - } - // 4. Decode the single Select (or default to documents). - let select = request_v1 - .selects - .first() - .cloned() - .map(conversions::select_from_proto) - .transpose()? - .unwrap_or_else(SelectProjection::documents); - // 5. HAVING decoding — wire-malformed clauses reject as - // `InvalidArgument` before any routing decision is taken. - let having = conversions::having_clauses_from_proto(request_v1.having.clone())?; - // 6. `validate_and_route` runs the inner `limit` / `having` / - // per-function gates. - let decision = validate_and_route( - &select, - request_v1.limit, - &having, - &request_v1.group_by, - &order_by_clauses, - where_clauses, - platform_version, - )?; - // 7. OFFSET, now that routing is known. - reject_offset_off_the_ranked_path(request_v1.offset, &decision)?; - Ok(match decision { - RoutingDecision::Documents => "documents", - RoutingDecision::Count(CountMode::Aggregate) => "count_aggregate", - RoutingDecision::Count(CountMode::GroupByIn) => "count_entries_via_in_field", - RoutingDecision::Count(CountMode::GroupByRange) => "count_entries_via_range_field", - RoutingDecision::Count(CountMode::GroupByCompound) => "count_entries_via_compound", - // v3 sum surface — single label for now (no sub-mode - // breakdown like count's). `dispatch_sum_v1` further routes - // by where-shape × prove flag. - RoutingDecision::Sum { .. } => "sum", - // v3 average surface — single label like sum; - // `dispatch_average_v1` further routes by where-shape × - // prove flag once the executor lands. - RoutingDecision::Average { .. } => "average", - // Ranked surface — single label; the axis / direction / `k` - // breakdown is drive's to resolve, not routing's, so there - // is no sub-mode to report here. - RoutingDecision::Ranked => "ranked", - }) + /// Boolean-`HAVING` range routing: a `COUNT` / `SUM` / `AVG` select + /// with a `GROUP BY` carrying exactly one `having` clause. Routing + /// does not read the clause — whether it bounds the selected + /// aggregate, whether its operator translates to a contiguous + /// range, and the limit's contract are drive's to resolve and to + /// refuse (`detect_having_mode`). Dispatches to + /// [`Self::dispatch_having_v1`] → + /// `Drive::execute_document_having_request` and emits the + /// `RankedEntries` proto message with `skipped` unset (a range page + /// has no rank base). Carries nothing, for the same reason `Ranked` + /// carries nothing. + HavingRange, } impl Platform { @@ -543,7 +165,9 @@ impl Platform { // once `validate_and_route` has answered, in // `reject_offset_off_the_ranked_path`. Off the ranked path the // rejection is byte-identical to the one this block used to - // emit. + // emit, except on the having-range route, which gets its own + // message (the legacy one recommends cursors that route also + // rejects). // Decode the proto-typed `repeated WhereClause` / `repeated // OrderClause` into drive's structured forms once, up @@ -678,885 +302,23 @@ impl Platform { platform_state, platform_version, ), + RoutingDecision::HavingRange => self.dispatch_having_v1( + data_contract_id, + document_type, + select, + group_by, + having_clauses, + where_clauses, + order_by_clauses, + limit, + offset, + start, + prove, + platform_state, + platform_version, + ), } } - - /// Dispatch a `select = SUM(field)` request to - /// [`Drive::execute_document_sum_request`] and map the response - /// into a `GetDocumentsResponseV1` carrying a `SumResults` payload - /// (or a `Proof` payload when prove=true). - /// - /// Parallels [`Self::dispatch_count_v1`] line-by-line — same - /// request construction, same error → typed-rejection mapping, - /// same prove vs no-prove split. Only the response shape mapping - /// differs: `DocumentSumResponse::Aggregate(i64)` → - /// `SumResults::aggregate_sum`, `Entries(Vec)` → - /// `SumResults::entries`, `Proof(bytes)` → outer `result.proof`. - #[allow(clippy::too_many_arguments)] - fn dispatch_sum_v1( - &self, - data_contract_id: Vec, - document_type_name: String, - where_clauses: Vec, - order_clauses: Vec, - limit: Option, - start: Option, - prove: bool, - sum_property: String, - mode: CountMode, - platform_state: &PlatformState, - platform_version: &PlatformVersion, - ) -> Result, Error> { - if start.is_some() { - return Ok(QueryValidationResult::new_with_error(not_yet_implemented( - "start_after / start_at with SELECT SUM (paginate by narrowing the \ - range clause itself)", - ))); - } - - let contract_id: Identifier = - check_validation_result_with_data!(data_contract_id.try_into().map_err(|_| { - QueryError::InvalidArgument( - "id must be a valid identifier (32 bytes long)".to_string(), - ) - })); - - let (_, contract_fetch_info) = self.drive.get_contract_with_fetch_info_and_fee( - contract_id.to_buffer(), - None, - true, - None, - platform_version, - )?; - let contract_fetch_info = check_validation_result_with_data!(contract_fetch_info.ok_or( - QueryError::Query(QuerySyntaxError::DataContractNotFound( - "contract not found when querying from value with contract info", - )) - )); - let contract_ref = &contract_fetch_info.contract; - let document_type = check_validation_result_with_data!(contract_ref - .document_type_for_name(document_type_name.as_str()) - .map_err(|_| QueryError::InvalidArgument(format!( - "document type {} not found for contract {}", - document_type_name, contract_id - )))); - - // `SumMode` mirrors `CountMode` 1:1 — same four variants - // computed via the same `compute_aggregate_mode_and_check_limit` - // helper. Map across the isomorphism. - let sum_mode = match mode { - CountMode::Aggregate => SumMode::Aggregate, - CountMode::GroupByIn => SumMode::GroupByIn, - CountMode::GroupByRange => SumMode::GroupByRange, - CountMode::GroupByCompound => SumMode::GroupByCompound, - }; - - let drive_request = DocumentSumRequest { - contract: contract_ref, - document_type, - sum_property, - where_clauses, - order_clauses, - mode: sum_mode, - limit, - prove, - drive_config: &self.config.drive, - }; - let drive_response = - match self - .drive - .execute_document_sum_request(drive_request, None, platform_version) - { - Ok(r) => r, - Err(drive::error::Error::Query(qe)) => { - return Ok(QueryValidationResult::new_with_error(QueryError::Query(qe))); - } - Err(e) => return Err(e.into()), - }; - - let response = match drive_response { - DocumentSumResponse::Aggregate(sum) => GetDocumentsResponseV1 { - result: Some(get_documents_response_v1::Result::Data(ResultData { - variant: Some(result_data::Variant::Sums(SumResults { - variant: Some(sum_results::Variant::AggregateSum(sum)), - })), - })), - metadata: Some(self.response_metadata_v0(platform_state, CheckpointUsed::Current)), - }, - DocumentSumResponse::Entries(entries) => { - if sum_mode == SumMode::Aggregate { - // Mirror of count's same-arm: `select=SUM, - // group_by=[]` whose executor routed through a - // PerInValue path (In + no range + no prove) - // returns one entry per In branch. Fold them into - // a single aggregate. `checked_add` surfaces the - // narrow case where per-branch sums truly add to - // more than i64::MAX as a typed - // `QuerySyntaxError::Unsupported` rather than - // silently saturating at i64::MAX (which produces - // a deterministic-but-misleading answer). - let mut total: i64 = 0; - let mut overflow = false; - for e in &entries { - match total.checked_add(e.sum.unwrap_or(0)) { - Some(t) => total = t, - None => { - overflow = true; - break; - } - } - } - if overflow { - return Ok(QueryValidationResult::new_with_error(QueryError::Query( - QuerySyntaxError::Unsupported( - "aggregate SUM across In branches overflows i64 — \ - the In-fold cannot be represented; narrow the In set \ - or query branches individually" - .to_string(), - ), - ))); - } - GetDocumentsResponseV1 { - result: Some(get_documents_response_v1::Result::Data(ResultData { - variant: Some(result_data::Variant::Sums(SumResults { - variant: Some(sum_results::Variant::AggregateSum(total)), - })), - })), - metadata: Some( - self.response_metadata_v0(platform_state, CheckpointUsed::Current), - ), - } - } else { - GetDocumentsResponseV1 { - result: Some(get_documents_response_v1::Result::Data(ResultData { - variant: Some(result_data::Variant::Sums(SumResults { - variant: Some(sum_results::Variant::Entries(SumEntries { - entries: entries.into_iter().map(into_v1_sum_entry).collect(), - })), - })), - })), - metadata: Some( - self.response_metadata_v0(platform_state, CheckpointUsed::Current), - ), - } - } - } - DocumentSumResponse::Proof(proof_bytes) => { - let (grovedb_used, proof) = - self.response_proof_v0(platform_state, proof_bytes, GroveDBToUse::Current)?; - GetDocumentsResponseV1 { - result: Some(get_documents_response_v1::Result::Proof(proof)), - metadata: Some(self.response_metadata_v0(platform_state, grovedb_used)), - } - } - }; - - Ok(QueryValidationResult::new_with_data(response)) - } - - /// Dispatch a `select = AVG(field)` request to - /// [`Drive::execute_document_average_request`] and map the response - /// into a `GetDocumentsResponseV1` carrying an `AverageResults` - /// payload (or a `Proof` payload when prove=true). - /// - /// Parallels [`Self::dispatch_sum_v1`] line-by-line — same request - /// construction, same error → typed-rejection mapping, same prove - /// vs no-prove split. The response shape mapping differs: - /// `DocumentAverageResponse::Aggregate { count, sum }` → - /// `AverageResults::aggregate_average`, - /// `DocumentAverageResponse::Entries(_)` → `AverageResults::entries`, - /// `DocumentAverageResponse::Proof(_)` → outer `result.proof`. - #[allow(clippy::too_many_arguments)] - fn dispatch_average_v1( - &self, - data_contract_id: Vec, - document_type_name: String, - where_clauses: Vec, - order_clauses: Vec, - limit: Option, - start: Option, - prove: bool, - sum_property: String, - mode: CountMode, - platform_state: &PlatformState, - platform_version: &PlatformVersion, - ) -> Result, Error> { - if start.is_some() { - return Ok(QueryValidationResult::new_with_error(not_yet_implemented( - "start_after / start_at with SELECT AVG (paginate by narrowing the \ - range clause itself)", - ))); - } - - let contract_id: Identifier = - check_validation_result_with_data!(data_contract_id.try_into().map_err(|_| { - QueryError::InvalidArgument( - "id must be a valid identifier (32 bytes long)".to_string(), - ) - })); - - let (_, contract_fetch_info) = self.drive.get_contract_with_fetch_info_and_fee( - contract_id.to_buffer(), - None, - true, - None, - platform_version, - )?; - let contract_fetch_info = check_validation_result_with_data!(contract_fetch_info.ok_or( - QueryError::Query(QuerySyntaxError::DataContractNotFound( - "contract not found when querying from value with contract info", - )) - )); - let contract_ref = &contract_fetch_info.contract; - let document_type = check_validation_result_with_data!(contract_ref - .document_type_for_name(document_type_name.as_str()) - .map_err(|_| QueryError::InvalidArgument(format!( - "document type {} not found for contract {}", - document_type_name, contract_id - )))); - - // `AverageMode` mirrors `CountMode` 1:1 — map across. - let avg_mode = match mode { - CountMode::Aggregate => AverageMode::Aggregate, - CountMode::GroupByIn => AverageMode::GroupByIn, - CountMode::GroupByRange => AverageMode::GroupByRange, - CountMode::GroupByCompound => AverageMode::GroupByCompound, - }; - - let drive_request = DocumentAverageRequest { - contract: contract_ref, - document_type, - sum_property, - where_clauses, - order_clauses, - mode: avg_mode, - limit, - prove, - drive_config: &self.config.drive, - }; - let drive_response = - match self - .drive - .execute_document_average_request(drive_request, None, platform_version) - { - Ok(r) => r, - Err(drive::error::Error::Query(qe)) => { - return Ok(QueryValidationResult::new_with_error(QueryError::Query(qe))); - } - Err(e) => return Err(e.into()), - }; - - let response = match drive_response { - DocumentAverageResponse::Aggregate { count, sum } => GetDocumentsResponseV1 { - result: Some(get_documents_response_v1::Result::Data(ResultData { - variant: Some(result_data::Variant::Averages(AverageResults { - variant: Some(average_results::Variant::AggregateAverage( - AverageAggregate { count, sum }, - )), - })), - })), - metadata: Some(self.response_metadata_v0(platform_state, CheckpointUsed::Current)), - }, - DocumentAverageResponse::Entries(entries) => { - if avg_mode == AverageMode::Aggregate { - // Mirror sum-side's fold for the `select=AVG, - // group_by=[]` + PerInValue executor combo. Fold - // both count and sum across In branches. Either - // axis overflowing is surfaced as a typed - // `QuerySyntaxError::Unsupported` so the client - // doesn't get a silently-saturated answer to - // divide against (which would also misreport the - // average). - let mut total_count: u64 = 0; - let mut total_sum: i64 = 0; - let mut overflow_axis: Option<&'static str> = None; - for e in &entries { - match total_count.checked_add(e.count.unwrap_or(0)) { - Some(c) => total_count = c, - None => { - overflow_axis = Some("count"); - break; - } - } - match total_sum.checked_add(e.sum.unwrap_or(0)) { - Some(s) => total_sum = s, - None => { - overflow_axis = Some("sum"); - break; - } - } - } - if let Some(axis) = overflow_axis { - return Ok(QueryValidationResult::new_with_error(QueryError::Query( - QuerySyntaxError::Unsupported(format!( - "aggregate AVG across In branches overflows {axis} \ - ({} axis range); narrow the In set or query branches \ - individually", - if axis == "count" { "u64" } else { "i64" }, - )), - ))); - } - GetDocumentsResponseV1 { - result: Some(get_documents_response_v1::Result::Data(ResultData { - variant: Some(result_data::Variant::Averages(AverageResults { - variant: Some(average_results::Variant::AggregateAverage( - AverageAggregate { - count: total_count, - sum: total_sum, - }, - )), - })), - })), - metadata: Some( - self.response_metadata_v0(platform_state, CheckpointUsed::Current), - ), - } - } else { - GetDocumentsResponseV1 { - result: Some(get_documents_response_v1::Result::Data(ResultData { - variant: Some(result_data::Variant::Averages(AverageResults { - variant: Some(average_results::Variant::Entries(AverageEntries { - entries: entries - .into_iter() - .map(into_v1_average_entry) - .collect(), - })), - })), - })), - metadata: Some( - self.response_metadata_v0(platform_state, CheckpointUsed::Current), - ), - } - } - } - DocumentAverageResponse::Proof(proof_bytes) => { - let (grovedb_used, proof) = - self.response_proof_v0(platform_state, proof_bytes, GroveDBToUse::Current)?; - GetDocumentsResponseV1 { - result: Some(get_documents_response_v1::Result::Proof(proof)), - metadata: Some(self.response_metadata_v0(platform_state, grovedb_used)), - } - } - }; - - Ok(QueryValidationResult::new_with_data(response)) - } - - /// Forward a `select = DOCUMENTS` request through the shared - /// `query_documents_typed` helper that v0 also dispatches into. - /// v1 doesn't add any documents-side capability — the SQL-shaped - /// fields (`select`, `group_by`, `having`) are all validated as - /// documents-compatible above (empty `group_by`, empty `having`, - /// etc.) before reaching here. - #[allow(clippy::too_many_arguments)] - fn dispatch_documents_v1( - &self, - data_contract_id: Vec, - document_type: String, - where_clauses: Vec, - order_by_clauses: Vec, - limit: Option, - start: Option, - prove: bool, - platform_state: &PlatformState, - platform_version: &PlatformVersion, - ) -> Result, Error> { - let start = start.map(|s| match s { - RequestV1Start::StartAfter(b) => RequestV0Start::StartAfter(b), - RequestV1Start::StartAt(b) => RequestV0Start::StartAt(b), - }); - // `limit` is `optional uint32` on v1; the typed helper takes - // `Option` directly (`None` → server default). `Some(0)` - // can't reach here — `validate_and_route` rejects it for - // every SELECT mode so the v1 contract is uniform; only - // `None` or `Some(N > 0)` survive. - let result = self.query_documents_typed( - data_contract_id, - document_type, - where_clauses, - order_by_clauses, - limit, - prove, - start, - platform_state, - platform_version, - )?; - Ok(result.map(translate_documents_v0_to_v1)) - } - - /// Forward a `select = COUNT` request to drive's count - /// dispatcher. `mode` is the SQL-shape contract derived from - /// `(select, group_by, where)` by `validate_and_route`; drive - /// uses it to pick the executor strategy and decide whether to - /// collapse the response to a single aggregate or return per- - /// group entries. The wire response is `GetDocumentsResponseV1` - /// with the inner `ResultData.counts` variant for non-proof - /// results. - #[allow(clippy::too_many_arguments)] - fn dispatch_count_v1( - &self, - data_contract_id: Vec, - document_type_name: String, - where_clauses: Vec, - order_clauses: Vec, - limit: Option, - start: Option, - prove: bool, - mode: CountMode, - platform_state: &PlatformState, - platform_version: &PlatformVersion, - ) -> Result, Error> { - if start.is_some() { - return Ok(QueryValidationResult::new_with_error(not_yet_implemented( - "start_after / start_at with SELECT COUNT (paginate by narrowing the \ - range clause itself)", - ))); - } - - let contract_id: Identifier = - check_validation_result_with_data!(data_contract_id.try_into().map_err(|_| { - QueryError::InvalidArgument( - "id must be a valid identifier (32 bytes long)".to_string(), - ) - })); - - let (_, contract_fetch_info) = self.drive.get_contract_with_fetch_info_and_fee( - contract_id.to_buffer(), - None, - true, - None, - platform_version, - )?; - let contract_fetch_info = check_validation_result_with_data!(contract_fetch_info.ok_or( - QueryError::Query(QuerySyntaxError::DataContractNotFound( - "contract not found when querying from value with contract info", - )) - )); - let contract_ref = &contract_fetch_info.contract; - let document_type = check_validation_result_with_data!(contract_ref - .document_type_for_name(document_type_name.as_str()) - .map_err(|_| QueryError::InvalidArgument(format!( - "document type {} not found for contract {}", - document_type_name, contract_id - )))); - - let drive_request = DocumentCountRequest { - contract: contract_ref, - document_type, - where_clauses, - order_clauses, - mode, - limit, - prove, - drive_config: &self.config.drive, - }; - let drive_response = - match self - .drive - .execute_document_count_request(drive_request, None, platform_version) - { - Ok(r) => r, - Err(drive::error::Error::Query(qe)) => { - return Ok(QueryValidationResult::new_with_error(QueryError::Query(qe))); - } - Err(e) => return Err(e.into()), - }; - - let response = match drive_response { - DocumentCountResponse::Aggregate(count) => GetDocumentsResponseV1 { - result: Some(get_documents_response_v1::Result::Data(ResultData { - variant: Some(result_data::Variant::Counts(CountResults { - variant: Some(count_results::Variant::AggregateCount(count)), - })), - })), - metadata: Some(self.response_metadata_v0(platform_state, CheckpointUsed::Current)), - }, - DocumentCountResponse::Entries(entries) => { - if mode.is_aggregate() { - // `select=COUNT, group_by=[]` against a request - // that drove a PerInValue execution (In + no - // range + no prove). Sum entries into a single - // aggregate before emission. `checked_add` - // surfaces u64 overflow as a typed - // `QuerySyntaxError::Unsupported`; realistic - // ceiling is `|In| × max_per-branch-count` (well - // under u64), so triggering this path requires - // either a misconfigured count tree or an - // executor bug. - let mut total: u64 = 0; - let mut overflow = false; - for e in &entries { - // `count.unwrap_or(0)` here is safe: this - // arm is server-side, summing entries the - // executor emitted. Executor never emits - // `None` (that's an SDK-side - // synthesis-for-missing concept). The - // `unwrap_or(0)` is a belt-and-suspenders - // guard against any future executor that - // forgets the contract. - match total.checked_add(e.count.unwrap_or(0)) { - Some(t) => total = t, - None => { - overflow = true; - break; - } - } - } - if overflow { - return Ok(QueryValidationResult::new_with_error(QueryError::Query( - QuerySyntaxError::Unsupported( - "aggregate COUNT across In branches overflows u64 — \ - narrow the In set or query branches individually" - .to_string(), - ), - ))); - } - GetDocumentsResponseV1 { - result: Some(get_documents_response_v1::Result::Data(ResultData { - variant: Some(result_data::Variant::Counts(CountResults { - variant: Some(count_results::Variant::AggregateCount(total)), - })), - })), - metadata: Some( - self.response_metadata_v0(platform_state, CheckpointUsed::Current), - ), - } - } else { - GetDocumentsResponseV1 { - result: Some(get_documents_response_v1::Result::Data(ResultData { - variant: Some(result_data::Variant::Counts(CountResults { - variant: Some(count_results::Variant::Entries(CountEntries { - entries: entries.into_iter().map(into_v1_entry).collect(), - })), - })), - })), - metadata: Some( - self.response_metadata_v0(platform_state, CheckpointUsed::Current), - ), - } - } - } - DocumentCountResponse::Proof(proof_bytes) => { - let (grovedb_used, proof) = - self.response_proof_v0(platform_state, proof_bytes, GroveDBToUse::Current)?; - GetDocumentsResponseV1 { - result: Some(get_documents_response_v1::Result::Proof(proof)), - metadata: Some(self.response_metadata_v0(platform_state, grovedb_used)), - } - } - }; - - Ok(QueryValidationResult::new_with_data(response)) - } - - /// Dispatch a ranked request — a `COUNT` / `SUM` / `AVG` select - /// with a `GROUP BY` whose single `ORDER BY` clause names the - /// selected aggregate — to - /// [`Drive::execute_document_ranked_request`], and map the response - /// into a `GetDocumentsResponseV1` carrying a `RankedEntries` - /// payload (or a `Proof` payload when prove=true). - /// - /// Structurally parallel to [`Self::dispatch_count_v1`] — same - /// contract fetch, same `Error::Query` → typed-rejection mapping, - /// same prove vs no-prove split — with two differences worth - /// naming: - /// - /// 1. **The request is forwarded whole.** `where_clauses`, - /// `having`, `order_by`, `limit`, `offset` and `start` all go - /// down, including the ones a ranked request must leave empty, - /// because drive owns those rejections: the SDK's client-side - /// helpers call drive's validator with no abci in the path, so - /// re-checking here would create a second, driftable copy of - /// the grammar. The rejections come back as `Error::Query(...)` - /// and are surfaced to the caller as query errors, not internal - /// ones. `order_by` in particular is no longer refused here — - /// it is the ranking. - /// 2. **Proving an empty ranking is mapped, not propagated.** See - /// [`empty_ranking_proof_rejection`]. - #[allow(clippy::too_many_arguments)] - fn dispatch_ranked_v1( - &self, - data_contract_id: Vec, - document_type_name: String, - select: SelectProjection, - group_by: Vec, - having: Vec, - where_clauses: Vec, - order_clauses: Vec, - limit: Option, - offset: Option, - start: Option, - prove: bool, - platform_state: &PlatformState, - platform_version: &PlatformVersion, - ) -> Result, Error> { - let contract_id: Identifier = - check_validation_result_with_data!(data_contract_id.try_into().map_err(|_| { - QueryError::InvalidArgument( - "id must be a valid identifier (32 bytes long)".to_string(), - ) - })); - - let (_, contract_fetch_info) = self.drive.get_contract_with_fetch_info_and_fee( - contract_id.to_buffer(), - None, - true, - None, - platform_version, - )?; - let contract_fetch_info = check_validation_result_with_data!(contract_fetch_info.ok_or( - QueryError::Query(QuerySyntaxError::DataContractNotFound( - "contract not found when querying from value with contract info", - )) - )); - let contract_ref = &contract_fetch_info.contract; - let document_type = check_validation_result_with_data!(contract_ref - .document_type_for_name(document_type_name.as_str()) - .map_err(|_| QueryError::InvalidArgument(format!( - "document type {} not found for contract {}", - document_type_name, contract_id - )))); - - let drive_request = DocumentRankedRequest { - contract: contract_ref, - document_type, - group_by: &group_by, - select, - having: &having, - order_by: &order_clauses, - where_clauses: &where_clauses, - limit, - offset, - has_start_at: start.is_some(), - prove, - }; - - let drive_response = - match self - .drive - .execute_document_ranked_request(drive_request, None, platform_version) - { - Ok(r) => r, - Err(drive::error::Error::Query(qe)) => { - return Ok(QueryValidationResult::new_with_error(QueryError::Query(qe))); - } - Err(e) => match empty_ranking_proof_rejection(&e) { - Some(rejection) => { - return Ok(QueryValidationResult::new_with_error(rejection)); - } - None => return Err(e.into()), - }, - }; - - let response = match drive_response { - DocumentRankedResponse::Entries(page) => GetDocumentsResponseV1 { - result: Some(get_documents_response_v1::Result::Data(ResultData { - // No aggregate-collapse arm here, unlike count / - // sum / average: a ranked result is always a list - // of groups. Even `LIMIT 1` returns one *entry*, - // because the caller needs to know which group - // won, not only the winning value. - variant: Some(result_data::Variant::Ranked(RankedEntries { - // Order is preserved verbatim: entry order is - // the ranking order, and drive already - // asserted the list is no longer than `k`. - entries: page.entries.into_iter().map(into_v1_ranked_entry).collect(), - // The page's starting rank, so entry `i` is - // identifiable as the group at rank - // `skipped + i` rather than as "one of the - // top few". This is the *unproven* path, so - // the number is only as good as the node — - // which is exactly why a proving client - // ignores it and re-derives the attested - // value from the proof bytes instead (see - // `RankedPage::skipped`). Sent as `Some` - // unconditionally, including the `0` an - // offset-less query produces: the proto field - // is `optional` to keep "this node predates - // the field" distinguishable from "this page - // starts at rank 0", and collapsing 0 to - // `None` would throw that distinction away. - skipped: Some(page.skipped), - })), - })), - metadata: Some(self.response_metadata_v0(platform_state, CheckpointUsed::Current)), - }, - DocumentRankedResponse::Proof(proof_bytes) => { - let (grovedb_used, proof) = - self.response_proof_v0(platform_state, proof_bytes, GroveDBToUse::Current)?; - GetDocumentsResponseV1 { - result: Some(get_documents_response_v1::Result::Proof(proof)), - metadata: Some(self.response_metadata_v0(platform_state, grovedb_used)), - } - } - }; - - Ok(QueryValidationResult::new_with_data(response)) - } -} - -/// Translate an rs-drive `RankedEntry` into the wire `RankedEntry`. -/// Mirror of [`into_v1_entry`] / [`into_v1_sum_entry`] / -/// [`into_v1_average_entry`] for the ranked surface. -/// -/// `key` passes through as the raw index-key bytes of the grouped -/// property value — the same bytes the proof commits to, so a client -/// verifying the proof compares byte-for-byte against what it -/// reconstructs. -/// -/// The `value` oneof is always set: drive's `RankedEntryValue` has no -/// "absent" variant (unlike the count / sum entry types, whose -/// `Option` exists for the SDK's synthesize-for-missing-In-value -/// concept — a ranked result has no caller-supplied key set to be -/// silent about). -fn into_v1_ranked_entry(e: DriveRankedEntry) -> RankedEntry { - RankedEntry { - key: e.key, - value: Some(match e.value { - RankedEntryValue::Count(count) => ranked_entry::Value::Count(count), - RankedEntryValue::Sum(sum) => ranked_entry::Value::Sum(sum), - // The wire carries a `double` approximation of the exact - // fixed-point `i128` the Avg axis is ordered by: - // `fixed_point as f64 / RANKED_AVG_SCALE as f64`, which is - // what `as_f64` computes. Lossy by construction, and that - // is fine — these entries are only read on the no-proof - // ("quick answer") path. A proof-verifying client ignores - // this field and reconstructs the exact fixed point from - // the grovedb proof, so no verification depends on it. - // Ranking order is still exact: the ordering happened over - // the i128 before this conversion. - value @ RankedEntryValue::AvgFixedPoint(_) => ranked_entry::Value::Avg(value.as_f64()), - }), - } -} - -/// Recognize the one grovedb failure that is a caller-facing -/// condition rather than a server fault: **an empty ranking cannot be -/// proved**. -/// -/// **This is now a backstop rather than a live path.** The ranked -/// prover moved to `prove_indexed_axis_top_k_paginated`, which emits a -/// guaranteed-empty range against an empty axis secondary instead of -/// refusing, so proving a ranking over a contract with no documents -/// succeeds and the proved and unproven paths agree (pinned by -/// `ranked_tests::proving_an_empty_ranking_succeeds`). The mapping is -/// kept because the failure it recognizes is a *class* — a merk-level -/// "cannot prove an empty tree" surfacing from somewhere in the -/// ancestor chain — not a single call site, and because the cost of -/// keeping it is one string comparison on an error path. -/// -/// Historically: the non-paginated prover had no absence-proof shape -/// for "this axis secondary has no entries", so proving a ranking over -/// an index that held no documents failed with a merk-level "Cannot -/// create proof for empty tree", wrapped by grovedb as -/// `CorruptedData`. Reaching that state needed nothing exotic — -/// querying a freshly registered contract with `prove = true` did it — -/// so letting it propagate would answer an ordinary request with an -/// internal error (`Status::unknown`) and an alarming server-side log -/// line, and give the caller no idea that the same request without -/// `prove` succeeded and returned the empty list. -/// -/// Detection is by variant + marker substring rather than by a typed -/// error, because grovedb flattens the merk error into a -/// `CorruptedData(String)` at the indexed-axis proof boundary; the -/// substring is the merk-side constant. The match is deliberately -/// narrow: any other `CorruptedData` still propagates as an internal -/// error, because for every other cause that classification is -/// correct. -fn empty_ranking_proof_rejection(error: &drive::error::Error) -> Option { - let drive::error::Error::GroveDB(grove_error) = error else { - return None; - }; - let drive::query::GroveError::CorruptedData(message) = grove_error.as_ref() else { - return None; - }; - if !message.contains("Cannot create proof for empty tree") { - return None; - } - Some(QueryError::InvalidArgument( - "this ranking has no groups yet, and an empty ranking cannot be proved: \ - grovedb has no absence-proof shape for an empty axis secondary. Retry \ - with `prove = false` — the unproven read answers the same request with \ - an empty entry list. Once the index holds at least one document, the \ - proved form works." - .to_string(), - )) -} - -fn into_v1_entry(e: SplitCountEntry) -> CountEntry { - CountEntry { - in_key: e.in_key, - key: e.key, - // The wire `count` is `uint64`, so it can only carry - // `Some(_)`. Server-side never emits `None` entries to - // begin with — `None` is the SDK-side synthesis signal for - // "caller's In array contained a value the proof was - // silent on," and that decision lives client-side because - // the wire never has the caller's full In array context. - // `unwrap_or(0)` is defense-in-depth: a future executor - // bug emitting `None` shouldn't crash the response path, - // it should round to zero on the wire (matching the - // proto's `uint64` default). - count: e.count.unwrap_or(0), - } -} - -/// Translate an rs-drive `SumEntry` into the wire `SumEntry`. Mirror -/// of [`into_v1_entry`] for the sum surface. -fn into_v1_sum_entry(e: DriveSumEntry) -> SumEntry { - SumEntry { - in_key: e.in_key, - key: e.key, - // `sum` is `sint64` on the wire — same `None`-rounds-to-0 - // contract as `into_v1_entry`. - sum: e.sum.unwrap_or(0), - } -} - -/// Translate an rs-drive `AverageEntry` into the wire `AverageEntry`. -/// Mirror of [`into_v1_entry`] + [`into_v1_sum_entry`] for the average -/// surface (carries both count and sum so the client can divide). -/// -/// `zip_entries` in `drive_document_average_query::drive_dispatcher` -/// performs a strict two-pointer merge that errors out as -/// `CorruptedCodeExecution` on any per-`(in_key, key)` divergence -/// between the count and sum streams. So by the time an entry reaches -/// this mapper, both axes have already been asserted to agree on -/// `Some`-vs-`None` for the same key — meaning the dangerous -/// `(count: None, sum: Some(V))` bucket that could let a client -/// divide V by 0 cannot exist. The `unwrap_or(0)` below is therefore -/// defense-in-depth (same as [`into_v1_entry`] / [`into_v1_sum_entry`] -/// for individual count / sum entries) rather than load-bearing. -fn into_v1_average_entry(e: DriveAverageEntry) -> AverageEntry { - AverageEntry { - in_key: e.in_key, - key: e.key, - count: e.count.unwrap_or(0), - sum: e.sum.unwrap_or(0), - } -} - -/// Translate a v0 `GetDocumentsResponseV0` into v1's response -/// envelope (Documents-or-Proof wrapping the v0 oneof result into -/// v1's `ResultData`-or-`Proof` shape). -fn translate_documents_v0_to_v1( - response_v0: dapi_grpc::platform::v0::get_documents_response::GetDocumentsResponseV0, -) -> GetDocumentsResponseV1 { - let metadata = response_v0.metadata; - let result = match response_v0.result { - Some(get_documents_response_v0::Result::Documents(docs)) => { - Some(get_documents_response_v1::Result::Data(ResultData { - variant: Some(result_data::Variant::Documents(Documents { - documents: docs.documents, - })), - })) - } - Some(get_documents_response_v0::Result::Proof(proof)) => { - Some(get_documents_response_v1::Result::Proof(proof)) - } - None => None, - }; - GetDocumentsResponseV1 { result, metadata } } #[cfg(test)] diff --git a/packages/rs-drive-abci/src/query/document_query/v1/routing.rs b/packages/rs-drive-abci/src/query/document_query/v1/routing.rs new file mode 100644 index 00000000000..3677b6bf0c3 --- /dev/null +++ b/packages/rs-drive-abci/src/query/document_query/v1/routing.rs @@ -0,0 +1,430 @@ +//! Request-shape routing for the v1 `getDocuments` handler: the +//! `select` × `group_by` × `order_by` × `having` supported-shape +//! table, and the post-routing offset gate. Split from `mod.rs` for +//! readability — the logic is unchanged and unversioned on its own: +//! the routing *rules* version through the +//! `compute_aggregate_mode_and_check_limit` table it calls into. + +use super::compute_aggregate_mode_and_check_limit::{ + compute_aggregate_mode_and_check_limit, AggregateRouting, +}; +use super::{not_yet_implemented, RoutingDecision}; +use crate::error::query::QueryError; +use dpp::version::PlatformVersion; +use drive::error::query::QuerySyntaxError; +use drive::query::{HavingClause, OrderClause, SelectFunction, SelectProjection, WhereClause}; +#[cfg(test)] +use { + super::conversions, dapi_grpc::platform::v0::get_documents_request::GetDocumentsRequestV1, + drive::query::CountMode, +}; + +/// Validate the `select` × `group_by` × `order_by` × `having` +/// combination against the supported-shape table (see the +/// message-level docstring on `GetDocumentsRequestV1` in +/// `platform.proto`). Returns the routing decision so the handler +/// knows whether to dispatch to the documents-fetch path, the count +/// path or the ranked path, and which response shape to produce. +#[allow(clippy::too_many_arguments)] +pub(super) fn validate_and_route( + select: &SelectProjection, + limit: Option, + having: &[HavingClause], + group_by: &[String], + order_by: &[OrderClause], + where_clauses: &[WhereClause], + platform_version: &PlatformVersion, +) -> Result { + // Centralized `limit: Some(0)` rejection. + // + // `limit` is `optional uint32` on the wire, so `Some(0)` is a + // distinct value any raw-gRPC/WASM/FFI caller can encode. Three + // legacy behaviors collide on this value across the v1 dispatch + // surface: + // - `SELECT DOCUMENTS` would `unwrap_or(0)` and forward to v0, + // where `limit=0` is the v0-uint32 sentinel for "use server + // default" — accept-as-default. + // - `SELECT COUNT` with `mode ∈ {Aggregate, GroupByIn}` would + // reject via the `is_some()` check below — reject-as-invalid. + // - `SELECT COUNT` with `mode ∈ {GroupByRange, GroupByCompound}` + // would pass `Some(0)` through to drive, which honors it as a + // zero-cap walk — accept-as-zero. + // + // Three semantics for the same wire bytes is bad contract. The + // v1 wire's whole point of switching to `optional uint32` was + // to make "unset" explicit (`None`), so `Some(0)` only makes + // sense as an *explicit* zero — and a zero-cap query returns + // no useful information regardless of mode. Reject it uniformly + // at the validation boundary so callers see a single, + // mode-independent contract: `None` for "use server default", + // `Some(N > 0)` for an explicit cap, `Some(0)` is invalid. + if limit == Some(0) { + return Err(QueryError::Query(QuerySyntaxError::InvalidLimit( + "limit = 0 is not a valid wire value on the v1 \ + `optional uint32` field; omit `limit` (None) to use the \ + server's default, or pass a positive integer for an \ + explicit cap (a zero-cap query is structurally \ + meaningless regardless of SELECT mode)" + .to_string(), + ))); + } + + // HAVING is only ever meaningful for an aggregate projection: + // it is a boolean predicate over the groups a `COUNT` / `SUM` / + // `AVG` produces. Those three functions route through the + // versioned `compute_aggregate_mode_and_check_limit` helper below, + // whose v2 table routes a single grouped clause to the + // having-range executor and whose older tables reject every + // non-empty HAVING, with wording that depends on whether the + // request is otherwise a ranked one. + // + // For every other SELECT there is no aggregate for a HAVING to + // talk about, so the rejection stays here and stays unversioned. + // It also stays *ahead* of the per-function gates below, exactly + // where the old blanket rejection was: `SELECT DOCUMENTS … HAVING` + // must keep reporting the HAVING rather than silently dropping it + // on the documents path, and `SELECT MIN/MAX … HAVING` must not + // start reporting MIN/MAX as the reason a request with two + // unsupported features was refused. + if !having.is_empty() + && !matches!( + select.function, + SelectFunction::Count | SelectFunction::Sum | SelectFunction::Avg + ) + { + return Err(not_yet_implemented("HAVING clause")); + } + + match select.function { + SelectFunction::Documents => { + if !select.field.is_empty() { + return Err(QueryError::InvalidArgument(format!( + "SELECT DOCUMENTS does not accept a projection field; \ + got field='{}' (omit the field for plain document fetch, \ + or use SELECT COUNT / SUM / AVG to project a value)", + select.field + ))); + } + if !group_by.is_empty() { + // GROUP BY with SELECT DOCUMENTS is structurally + // nonsensical — GROUP BY produces one row per + // distinct key, but SELECT DOCUMENTS returns the + // underlying rows; the two contracts can't be + // reconciled. Callers wanting per-group output use + // SELECT COUNT / SUM / AVG / MIN / MAX. Classify + // as `InvalidArgument` rather than + // `not_yet_implemented` because this isn't a + // future capability — no protocol version will + // make this combination meaningful. + return Err(QueryError::InvalidArgument(format!( + "GROUP BY with SELECT DOCUMENTS is not a valid SQL shape: \ + GROUP BY produces one row per distinct key, but SELECT \ + DOCUMENTS returns the underlying rows themselves. Use \ + SELECT COUNT / SUM / AVG / MIN / MAX with GROUP BY for \ + per-group output, or SELECT DOCUMENTS without GROUP BY \ + for plain document fetch. Got group_by={:?}.", + group_by + ))); + } + Ok(RoutingDecision::Documents) + } + SelectFunction::Sum => { + // SELECT SUM(field): routes to + // `Drive::execute_document_sum_request` (in + // `packages/rs-drive/src/query/drive_document_sum_query/`). + // `field` must be non-empty and must name an integer + // property on the document type that's covered by either + // `documents_summable` (doctype level) or a `summable: + // ""` index. Validation lives downstream in + // [`crate::query::drive_document_sum_query::drive_dispatcher::detect_sum_mode`]. + // + // Wiring: `RoutingDecision::Sum(...)` variant below feeds + // the dispatch arm in the response-building section, which + // routes the resulting `DocumentSumResponse` into the + // `SumResults` proto message defined in platform.proto. + if select.field.is_empty() { + return Err(QueryError::InvalidArgument( + "SELECT SUM requires a non-empty `field` naming the integer property \ + to sum (e.g. `SUM(amount)`). The contract must declare \ + `documentsSummable: \"\"` at the document-type level OR a \ + `summable: \"\"` index covering the where-clause shape; the \ + DPP validator enforces this at contract creation." + .to_string(), + )); + } + match compute_aggregate_mode_and_check_limit( + select, + group_by, + where_clauses, + order_by, + limit, + having, + "SUM", + platform_version, + )? { + AggregateRouting::Grouped(mode) => Ok(RoutingDecision::Sum { + sum_property: select.field.clone(), + mode, + }), + AggregateRouting::Ranked => Ok(RoutingDecision::Ranked), + AggregateRouting::HavingRange => Ok(RoutingDecision::HavingRange), + } + } + SelectFunction::Avg => { + // SELECT AVG(field): routes to + // `Drive::execute_document_average_request` (in + // `packages/rs-drive/src/query/drive_document_average_query/`). + // `field` must be non-empty and must name an integer + // property covered by either `documents_summable` (doctype + // level) or a `summable: ""` index — averages reuse + // sum-tree indexes (no separate `averageable` flag exists + // or is needed; the same `CountSumTree` / PCPS element + // backs both). + // + // Wiring: `RoutingDecision::Average(...)` variant below + // feeds the dispatch arm in the response-building section, + // which routes the resulting `DocumentAverageResponse` into + // the `AverageResults` proto message defined in + // platform.proto. + if select.field.is_empty() { + return Err(QueryError::InvalidArgument( + "SELECT AVG requires a non-empty `field` naming the integer property \ + to average (e.g. `AVG(score)`). The contract must declare \ + `documentsSummable: \"\"` at the document-type level OR a \ + `summable: \"\"` index covering the where-clause shape; the \ + DPP validator enforces this at contract creation. Averages reuse \ + sum-tree indexes — no separate `averageable` flag is required." + .to_string(), + )); + } + match compute_aggregate_mode_and_check_limit( + select, + group_by, + where_clauses, + order_by, + limit, + having, + "AVG", + platform_version, + )? { + AggregateRouting::Grouped(mode) => Ok(RoutingDecision::Average { + sum_property: select.field.clone(), + mode, + }), + AggregateRouting::Ranked => Ok(RoutingDecision::Ranked), + AggregateRouting::HavingRange => Ok(RoutingDecision::HavingRange), + } + } + SelectFunction::Min => Err(not_yet_implemented( + "SELECT MIN (the wire surface accepts MIN(field) so callers \ + can encode it ahead of server support landing, but the \ + server doesn't yet evaluate per-group MIN; semantically \ + distinct from asking for the lowest-ranked group, which is \ + `ORDER BY ASC LIMIT 1`)", + )), + SelectFunction::Max => Err(not_yet_implemented( + "SELECT MAX (the wire surface accepts MAX(field) so callers \ + can encode it ahead of server support landing, but the \ + server doesn't yet evaluate per-group MAX; semantically \ + distinct from asking for the highest-ranked group, which is \ + `ORDER BY DESC LIMIT 1`)", + )), + SelectFunction::Count => { + if !select.field.is_empty() { + return Err(not_yet_implemented( + "SELECT COUNT(field) — counting non-null values of a \ + specific field (the wire surface accepts the field so \ + callers can encode it ahead of server support landing, \ + but today only COUNT(*) — empty `field` — is evaluated)", + )); + } + // Field-membership predicates on the request's where + // clauses. **Match-any, not match-first** — a request + // may carry two range clauses on different fields + // (the executor's `RangeAggregateCarrierProof` path + // is built for exactly that shape; see + // `outer_range_plus_inner_range_with_prove_and_group_by_range_routes_to_carrier_proof` + // in `drive/query/drive_document_count_query/tests.rs`). + // A `find(...).map(field).map(eq)` test against a + // hard-coded first range clause would make the routing + // decision depend on clause ordering on the wire, + // which is wrong — `WHERE a > x AND b > y GROUP BY a` + // and `WHERE b > y AND a > x GROUP BY a` must produce + // the same routing. + // + // For `In` the practical effect is the same because + // `validate_and_canonicalize_where_clauses` rejects + // multiple `In` clauses upstream (`MultipleInClauses`), + // but the `any` shape is used here too so the routing + // logic doesn't bake in an assumption that could go + // stale if that validator's contract ever relaxes. + match compute_aggregate_mode_and_check_limit( + select, + group_by, + where_clauses, + order_by, + limit, + having, + "COUNT", + platform_version, + )? { + AggregateRouting::Grouped(mode) => Ok(RoutingDecision::Count(mode)), + AggregateRouting::Ranked => Ok(RoutingDecision::Ranked), + AggregateRouting::HavingRange => Ok(RoutingDecision::HavingRange), + } + } + } +} + +/// The `OFFSET` gate, applied **after** routing. +/// +/// Offset pagination exists on exactly one path: the ranked executor, +/// where `OFFSET m` is the rank the returned page starts at and costs +/// nothing to prove (grovedb attests the skipped region from counted +/// subtree commitments rather than walking it). Every other v1 shape — +/// documents, and the grouped count / sum / average modes — has no +/// offset primitive behind it and keeps the rejection it has always +/// had, **message for message**: those callers paginate with +/// `start_after` / `start_at`, or by narrowing the range clause. +/// +/// The legacy message below is load-bearing and must not be reworded: +/// clients match on it, and on a protocol version whose routing table +/// has no ranked path (v13 and earlier) it is the *only* answer an +/// offset can get, exactly as it was before the ranked surface existed. +/// +/// The having-range route gets its own message instead, because the +/// legacy one gives that caller wrong advice: the having surface has +/// neither offset nor cursor pagination (`start_after` / `start_at` +/// are rejected by mode detection — a document-ID cursor cannot +/// address the aggregate-sorted secondary). The only continuation is +/// tightening the bound past the last distinct aggregate value, with +/// the documented tie limitation. +pub(super) fn reject_offset_off_the_ranked_path( + offset: Option, + decision: &RoutingDecision, +) -> Result<(), QueryError> { + match decision { + _ if offset.is_none() => Ok(()), + RoutingDecision::Ranked => Ok(()), + RoutingDecision::HavingRange => Err(not_yet_implemented( + "OFFSET on a having-range query; this surface has no offset or cursor \ + pagination — to continue past a page cut at the limit, tighten the \ + `having` bound past the last aggregate value seen (this cannot cross \ + a tie: several groups sharing the boundary aggregate must fit inside \ + one limit)", + )), + _ => Err(not_yet_implemented( + "OFFSET pagination (use cursor pagination via `start_after` / \ + `start_at` instead)", + )), + } +} + +/// Test-only: expose the routing decision for unit tests without +/// needing a full `Platform` setup. Mirrors **both the rejection +/// messages and the gate ordering** of [`Platform::query_documents_v1`] +/// so a test that pins a first-fail message also pins the order +/// gates fire in, not just which gate eventually fires. +/// +/// Sequence (same as the real handler at +/// [`Platform::query_documents_v1`]): +/// 1. `where_clauses_from_proto` → propagate `InvalidArgument` / +/// `Unsupported` decode errors +/// 2. `order_clauses_from_proto` → propagate aggregate-target +/// rejection / `InvalidArgument` decode errors +/// 3. `selects.len() > 1` → `not_yet_implemented("multi-projection …")` +/// 4. `select_from_proto` (first element, or default documents) +/// 5. `having_clauses_from_proto` → propagate `InvalidArgument` +/// decode errors (unknown aggregate-function / operator +/// discriminant, missing aggregate, missing / retired right +/// operand) +/// 6. [`validate_and_route`] — which itself runs `limit == Some(0)` +/// → the non-aggregate HAVING gate → per-function gates → +/// routing pick (including the versioned ranked gate). +/// 7. `offset.is_some()` on a non-ranked decision → +/// `not_yet_implemented("OFFSET …")` +/// +/// The OFFSET gate is **last**, not first: whether an offset is +/// acceptable now depends on where the request routes (the ranked +/// executor paginates by offset; nothing else does), and that is not +/// known until routing has run. On a protocol version whose table has +/// no ranked path, every offset is still refused with the identical +/// message — only its position relative to the decode gates moved. +/// +/// Treats an unset `select` (proto-default) the same way the +/// handler does — as `SelectProjection::documents()`. +#[cfg(test)] +pub(super) fn validate_and_route_for_tests( + request_v1: &GetDocumentsRequestV1, + where_clauses: &[WhereClause], + platform_version: &PlatformVersion, +) -> Result<&'static str, QueryError> { + // 1. WHERE decoding — wire-malformed shapes (unknown operator + // discriminant, nested `DocumentFieldValue.list` beyond + // depth 1, …) reject as `InvalidArgument`. Runs even + // though the caller passes a separate pre-decoded + // `where_clauses` slice for the routing decision, because + // the depth-cap and similar decode-time contracts aren't + // exercisable otherwise. + conversions::where_clauses_from_proto(request_v1.where_clauses.clone())?; + // 2. ORDER BY decoding — aggregate-target reject as + // `Unsupported("ORDER BY on aggregate keys …")`. + let order_by_clauses = conversions::order_clauses_from_proto(request_v1.order_by.clone())?; + // 3. Multi-projection SELECT rejection. + if request_v1.selects.len() > 1 { + return Err(not_yet_implemented( + "multi-projection SELECT (the wire accepts `repeated Select` so \ + callers can encode `SELECT COUNT(*), SUM(amount), AVG(rating)` \ + ahead of server support landing, but today only single-projection \ + requests are evaluated; the response shape will gain a parallel \ + `repeated AggregateValue values` field when multi-projection \ + lands)", + )); + } + // 4. Decode the single Select (or default to documents). + let select = request_v1 + .selects + .first() + .cloned() + .map(conversions::select_from_proto) + .transpose()? + .unwrap_or_else(SelectProjection::documents); + // 5. HAVING decoding — wire-malformed clauses reject as + // `InvalidArgument` before any routing decision is taken. + let having = conversions::having_clauses_from_proto(request_v1.having.clone())?; + // 6. `validate_and_route` runs the inner `limit` / `having` / + // per-function gates. + let decision = validate_and_route( + &select, + request_v1.limit, + &having, + &request_v1.group_by, + &order_by_clauses, + where_clauses, + platform_version, + )?; + // 7. OFFSET, now that routing is known. + reject_offset_off_the_ranked_path(request_v1.offset, &decision)?; + Ok(match decision { + RoutingDecision::Documents => "documents", + RoutingDecision::Count(CountMode::Aggregate) => "count_aggregate", + RoutingDecision::Count(CountMode::GroupByIn) => "count_entries_via_in_field", + RoutingDecision::Count(CountMode::GroupByRange) => "count_entries_via_range_field", + RoutingDecision::Count(CountMode::GroupByCompound) => "count_entries_via_compound", + // v3 sum surface — single label for now (no sub-mode + // breakdown like count's). `dispatch_sum_v1` further routes + // by where-shape × prove flag. + RoutingDecision::Sum { .. } => "sum", + // v3 average surface — single label like sum; + // `dispatch_average_v1` further routes by where-shape × + // prove flag once the executor lands. + RoutingDecision::Average { .. } => "average", + // Ranked surface — single label; the axis / direction / `k` + // breakdown is drive's to resolve, not routing's, so there + // is no sub-mode to report here. + RoutingDecision::Ranked => "ranked", + // Having-range surface — single label for the same reason as + // ranked: the bounds / direction / limit breakdown is drive's. + RoutingDecision::HavingRange => "having_range", + }) +} diff --git a/packages/rs-drive-abci/src/query/document_query/v1/tests.rs b/packages/rs-drive-abci/src/query/document_query/v1/tests.rs index 860c394c18f..206e421a6ab 100644 --- a/packages/rs-drive-abci/src/query/document_query/v1/tests.rs +++ b/packages/rs-drive-abci/src/query/document_query/v1/tests.rs @@ -165,10 +165,12 @@ fn assert_not_yet_implemented(result: Result<&'static str, QueryError>, expected #[test] fn reject_having_non_empty() { - // Non-empty `having` is rejected wholesale until the server - // gains HAVING-evaluation capability. The clause shape itself - // doesn't matter (server doesn't decode it past the `is_empty()` - // check), so a single placeholder clause is sufficient. + // Non-empty `having` on a **non-aggregate** select stays rejected + // by the unversioned gate: this request carries no `selects`, so it + // defaults to `SELECT DOCUMENTS`, and there is no aggregate for a + // HAVING to talk about. (The aggregate selects route through the + // versioned helper, whose v2 table serves a single grouped clause — + // see `having_range_tests`.) let request = GetDocumentsRequestV1 { having: vec![hc( having_aggregate::Function::Count, @@ -1964,17 +1966,17 @@ mod ranked_tests { /// Shared with rs-drive's ranked suite — see the module docs for /// why this is a cross-crate path and not a copy. - const RESTAURANTS_CONTRACT_PATH: &str = + pub(super) const RESTAURANTS_CONTRACT_PATH: &str = "../rs-drive/tests/supporting_files/contract/restaurants/restaurants-contract.json"; /// The one property every fixture doctype groups by. - const GROUP_PROPERTY: &str = "restaurantId"; + pub(super) const GROUP_PROPERTY: &str = "restaurantId"; /// The last protocol version whose query table (v0) has no ranked /// path at all. Ranked routing activates at 14. - const PROTOCOL_VERSION_V13: u32 = 13; + pub(super) const PROTOCOL_VERSION_V13: u32 = 13; - fn register_restaurants( + pub(super) fn register_restaurants( platform: &Platform, platform_version: &PlatformVersion, ) -> DataContract { @@ -1992,7 +1994,7 @@ mod ranked_tests { /// `first_seed` seeds the random-document generator, which derives /// each document's id; two calls in one test need disjoint seed /// ranges or the second collides on an existing id. - fn insert_docs( + pub(super) fn insert_docs( platform: &Platform, contract: &DataContract, document_type_name: &str, @@ -2025,7 +2027,7 @@ mod ranked_tests { } } - fn select(function: v1_select::Function, field: &str) -> Vec { + pub(super) fn select(function: v1_select::Function, field: &str) -> Vec { vec![V1Select { function: function as i32, field: field.to_string(), @@ -2091,7 +2093,7 @@ mod ranked_tests { /// **page** — entries plus the `skipped` rank base — asserting the /// response landed on the `ranked` variant of `ResultData` rather /// than on `counts` / `sums` / `averages`. - fn ranked_page( + pub(super) fn ranked_page( platform: &Platform, state: &PlatformState, request: GetDocumentsRequestV1, @@ -2119,7 +2121,7 @@ mod ranked_tests { /// [`ranked_page`] for the majority of tests, which only care about /// the entries. - fn ranked_entries( + pub(super) fn ranked_entries( platform: &Platform, state: &PlatformState, request: GetDocumentsRequestV1, @@ -2133,7 +2135,7 @@ mod ranked_tests { /// validation result, which the gRPC layer turns into /// `invalid_argument` — and never as the `Err` arm, which becomes /// an opaque internal error. - fn ranked_error( + pub(super) fn ranked_error( platform: &Platform, state: &PlatformState, request: GetDocumentsRequestV1, @@ -2153,7 +2155,7 @@ mod ranked_tests { result.errors.into_iter().next().expect("checked non-empty") } - fn group_keys(entries: &[RankedEntry]) -> Vec { + pub(super) fn group_keys(entries: &[RankedEntry]) -> Vec { entries .iter() .map(|entry| String::from_utf8(entry.key.clone()).expect("fixture keys are utf-8")) @@ -2706,14 +2708,28 @@ mod ranked_tests { assert_eq!(sums(&bottom_one), vec![10]); } - /// A boolean `HAVING` alongside an aggregate ordering is refused - /// with the `not_yet_implemented` contract — a client can leave the - /// request in place and it starts working when the capability - /// lands. + /// A single boolean `HAVING` clause alongside an aggregate ordering + /// no longer rides the ranked path at all: the v2 routing helper + /// sends any grouped single-clause `having` to the having-range + /// surface, where an `ORDER BY` naming the selected aggregate is + /// legal and sets the walk direction. This request — a client left + /// in place across the capability landing, exactly as the old + /// `not_yet_implemented` contract promised — now answers. Full + /// having-range behaviour is pinned in [`super::having_range_tests`]; + /// this test pins the routing handoff from the ranked shape. #[test] - fn having_alongside_an_aggregate_ordering_is_still_unsupported() { + fn having_alongside_an_aggregate_ordering_now_routes_to_having_range() { let (platform, state, version) = setup_platform(None, Network::Testnet, None); let contract = register_restaurants(&platform, version); + insert_docs( + &platform, + &contract, + "review", + "grade", + 9_000, + &[("alpha", 90), ("beta", 30), ("gamma", 60)], + version, + ); let mut request = ranked_desc( &contract, @@ -2726,18 +2742,19 @@ mod ranked_tests { having_aggregate::Function::Avg, "grade", having_clause::Operator::GreaterThan, - Value::U64(4), + Value::U64(40), )]; - match ranked_error(&platform, &state, request, version) { - QueryError::Query(QuerySyntaxError::Unsupported(message)) => { - assert!( - message.contains("not yet implemented") && message.contains("HAVING"), - "expected the HAVING-with-ordering rejection, got: {message}" - ); - } - other => panic!("expected Unsupported, got {other:?}"), - } + let page = ranked_page(&platform, &state, request, version); + assert_eq!( + page.skipped, None, + "a having-range page has no rank base and must leave `skipped` unset" + ); + assert_eq!( + group_keys(&page.entries), + vec!["alpha", "gamma"], + "descending walk over averages above 40: alpha (90) then gamma (60)" + ); } /// Ordering by the selected aggregate **without** a `GROUP BY` is @@ -3170,3 +3187,870 @@ mod multi_in_wire_tests { } } } + +mod having_range_tests { + //! End-to-end coverage of the having-range + //! (`GROUP BY p HAVING LIMIT n`) + //! surface through the real v1 handler: wire request in, + //! `ResultData.ranked` with `skipped` unset (or a `Proof`) out. + //! + //! Shares the `restaurants` fixture with [`super::ranked_tests`] — + //! same cross-crate path, same doctype → axis table — because the + //! having-range surface reads the very same indexed trees; only the + //! addressing (value bound instead of rank) differs. Value-level + //! behaviour (bounds translation, proof round-trips, tamper + //! rejection) is pinned in rs-drive's + //! `drive_document_having_query::tests`; this suite pins the wire + //! encoding, the routing, and the rejection contracts. + + use super::ranked_tests::{ + group_keys, insert_docs, ranked_error, ranked_page, register_restaurants, select, + GROUP_PROPERTY, PROTOCOL_VERSION_V13, + }; + use super::*; + + /// The canonical having-range request: one aggregate select, one + /// `group_by`, one `having` clause on the selected aggregate, a + /// `limit`, and optionally an `order_by` naming the selected + /// aggregate. Everything else at its "unset" wire value. + #[allow(clippy::too_many_arguments)] + fn having_request( + contract: &dpp::prelude::DataContract, + document_type: &str, + selects: Vec, + clause: ProtoHavingClause, + order_by: Vec, + limit: Option, + prove: bool, + ) -> GetDocumentsRequestV1 { + GetDocumentsRequestV1 { + data_contract_id: contract.id().to_vec(), + document_type: document_type.to_string(), + where_clauses: Vec::new(), + order_by, + limit, + start: None, + prove, + selects, + group_by: vec![GROUP_PROPERTY.to_string()], + having: vec![clause], + offset: None, + } + } + + /// `SELECT COUNT(*) GROUP BY restaurantId HAVING $count > 2 + /// LIMIT 10` — the headline spam-resistant-discovery shape. No + /// `order_by`: ascending by count is the default, and `skipped` + /// must be unset because a value-bounded page has no rank base. + #[test] + fn count_threshold_returns_matching_entries_with_no_rank_base() { + let (platform, state, version) = setup_platform(None, Network::Testnet, None); + let contract = register_restaurants(&platform, version); + insert_docs( + &platform, + &contract, + "visit", + "guests", + 10_000, + &[ + ("alpha", 1), + ("beta", 1), + ("beta", 2), + ("beta", 3), + ("gamma", 1), + ("gamma", 2), + ("delta", 1), + ("delta", 2), + ("delta", 3), + ("delta", 4), + ], + version, + ); + + let request = having_request( + &contract, + "visit", + select(v1_select::Function::Count, ""), + hc( + having_aggregate::Function::Count, + "", + having_clause::Operator::GreaterThan, + Value::U64(2), + ), + Vec::new(), + Some(10), + false, + ); + + let page = ranked_page(&platform, &state, request, version); + assert_eq!( + page.skipped, None, + "a having-range page must leave the rank-based `skipped` field unset" + ); + assert_eq!( + group_keys(&page.entries), + vec!["beta", "delta"], + "ascending count order: beta (3 visits) before delta (4)" + ); + } + + /// `prove = true` answers with a `Proof` payload, exactly like the + /// ranked path. The proof's verifiability is pinned in rs-drive's + /// suite; here only the wire shape is asserted. + #[test] + fn a_having_request_with_prove_returns_a_proof() { + let (platform, state, version) = setup_platform(None, Network::Testnet, None); + let contract = register_restaurants(&platform, version); + insert_docs( + &platform, + &contract, + "visit", + "guests", + 11_000, + &[("alpha", 1), ("beta", 1), ("beta", 2), ("beta", 3)], + version, + ); + + let request = having_request( + &contract, + "visit", + select(v1_select::Function::Count, ""), + hc( + having_aggregate::Function::Count, + "", + having_clause::Operator::GreaterThan, + Value::U64(2), + ), + Vec::new(), + Some(10), + true, + ); + + let result = platform + .query_documents_v1(request, &state, version) + .expect("query call should not error at the transport layer"); + assert!( + result.errors.is_empty(), + "expected no validation errors, got {:?}", + result.errors + ); + match result.data { + Some(GetDocumentsResponseV1 { + result: Some(get_documents_response_v1::Result::Proof(_)), + metadata: Some(_), + }) => {} + other => panic!("expected a Proof result, got {:?}", other), + } + } + + /// Two clauses (implicit AND) keep the `not_yet_implemented` + /// contract, with a message that names the restriction rather than + /// the blanket "HAVING clause". + #[test] + fn multiple_clauses_are_still_not_implemented() { + let (platform, state, version) = setup_platform(None, Network::Testnet, None); + let contract = register_restaurants(&platform, version); + + let clause = hc( + having_aggregate::Function::Count, + "", + having_clause::Operator::GreaterThan, + Value::U64(2), + ); + let mut request = having_request( + &contract, + "visit", + select(v1_select::Function::Count, ""), + clause.clone(), + Vec::new(), + Some(10), + false, + ); + request.having.push(clause); + + match ranked_error(&platform, &state, request, version) { + QueryError::Query(QuerySyntaxError::Unsupported(message)) => { + assert!( + message.contains("multiple HAVING clauses") + && message.contains("not yet implemented"), + "expected the multi-clause rejection, got: {message}" + ); + } + other => panic!("expected Unsupported, got {other:?}"), + } + } + + /// `GROUP BY restaurantId, guests HAVING …` — the routing layer + /// sends any grouped single-clause having down the having path + /// (it owns *where* the request goes, not the grammar), and + /// drive's mode detection rejects the compound grouping: ranked + /// axes live on single-property indexes. + #[test] + fn compound_group_by_is_rejected_on_the_having_path() { + let (platform, state, version) = setup_platform(None, Network::Testnet, None); + let contract = register_restaurants(&platform, version); + + let mut request = having_request( + &contract, + "visit", + select(v1_select::Function::Count, ""), + hc( + having_aggregate::Function::Count, + "", + having_clause::Operator::GreaterThan, + Value::U64(2), + ), + Vec::new(), + Some(10), + false, + ); + request.group_by = vec![GROUP_PROPERTY.to_string(), "guests".to_string()]; + + match ranked_error(&platform, &state, request, version) { + QueryError::Query(QuerySyntaxError::InvalidParameter(message)) => { + assert!( + message.contains("exactly one `group_by` property"), + "the rejection must say the surface is single-property, got: {message}" + ); + } + other => panic!("expected InvalidParameter, got {other:?}"), + } + } + + /// `OFFSET` stays ranked-only, and the having-range route gets its + /// own rejection: the legacy message recommends `start_after` / + /// `start_at`, which this surface also rejects, so the having + /// message explains continuation-by-bound instead of pointing at + /// an unsupported cursor. + #[test] + fn offset_is_rejected_on_the_having_path() { + let (platform, state, version) = setup_platform(None, Network::Testnet, None); + let contract = register_restaurants(&platform, version); + + let mut request = having_request( + &contract, + "visit", + select(v1_select::Function::Count, ""), + hc( + having_aggregate::Function::Count, + "", + having_clause::Operator::GreaterThan, + Value::U64(2), + ), + Vec::new(), + Some(10), + false, + ); + request.offset = Some(1); + + match ranked_error(&platform, &state, request, version) { + QueryError::Query(QuerySyntaxError::Unsupported(message)) => { + assert!( + message.contains("OFFSET on a having-range query"), + "expected the having-specific offset message, got: {message}" + ); + assert!( + message.contains("tighten the `having` bound"), + "the message must explain continuation-by-bound, got: {message}" + ); + assert!( + !message.contains("start_after"), + "the message must not recommend cursors this surface rejects, got: {message}" + ); + } + other => panic!("expected Unsupported, got {other:?}"), + } + } + + /// A bound on an axis the index does not declare surfaces as a + /// query error naming the missing contract keyword. The `review` + /// doctype's index is `rankedAverageable` only. + #[test] + fn a_bound_on_an_undeclared_axis_names_the_missing_keyword() { + let (platform, state, version) = setup_platform(None, Network::Testnet, None); + let contract = register_restaurants(&platform, version); + + let request = having_request( + &contract, + "review", + select(v1_select::Function::Count, ""), + hc( + having_aggregate::Function::Count, + "", + having_clause::Operator::GreaterThan, + Value::U64(2), + ), + Vec::new(), + Some(10), + false, + ); + + let error = ranked_error(&platform, &state, request, version); + assert!( + format!("{error}").contains("rankedCountable"), + "the rejection must name the missing keyword, got: {error}" + ); + } + + /// Non-contiguous operators (`!=`, `IN`) reach drive and are + /// refused there with a message explaining the contiguity + /// requirement — as a query error, never an internal one. + #[test] + fn non_contiguous_operators_surface_as_query_errors() { + let (platform, state, version) = setup_platform(None, Network::Testnet, None); + let contract = register_restaurants(&platform, version); + + for operator in [ + having_clause::Operator::NotEqual, + having_clause::Operator::In, + ] { + let request = having_request( + &contract, + "visit", + select(v1_select::Function::Count, ""), + hc( + having_aggregate::Function::Count, + "", + operator, + Value::U64(2), + ), + Vec::new(), + Some(10), + false, + ); + match ranked_error(&platform, &state, request, version) { + QueryError::Query(QuerySyntaxError::Unsupported(message)) => { + assert!( + message.contains("contiguous"), + "expected the contiguity rejection for {operator:?}, got: {message}" + ); + } + other => panic!("expected Unsupported for {operator:?}, got {other:?}"), + } + } + } + + /// Protocol version 13's query table (v0 helper) has no having + /// path: the same request a v14 node answers is refused with the + /// blanket rejection. The routing gate fires before any contract + /// fetch, so no ranked contract is needed (v13's meta-schema could + /// not register one anyway). + #[test] + fn protocol_version_13_still_rejects_having() { + let (platform, state, version) = + setup_platform(None, Network::Testnet, Some(PROTOCOL_VERSION_V13)); + + let request = GetDocumentsRequestV1 { + data_contract_id: vec![0u8; 32], + document_type: "visit".to_string(), + where_clauses: Vec::new(), + order_by: Vec::new(), + limit: Some(10), + start: None, + prove: false, + selects: select(v1_select::Function::Count, ""), + group_by: vec![GROUP_PROPERTY.to_string()], + having: vec![hc( + having_aggregate::Function::Count, + "", + having_clause::Operator::GreaterThan, + Value::U64(2), + )], + offset: None, + }; + + match ranked_error(&platform, &state, request, version) { + QueryError::Query(QuerySyntaxError::Unsupported(message)) => { + assert!( + message.contains("HAVING clause") && message.contains("not yet implemented"), + "expected v13's blanket rejection, got: {message}" + ); + } + other => panic!("expected Unsupported, got {other:?}"), + } + } + + /// The routing label: a grouped aggregate with one having clause + /// routes to `having_range`, and without the group_by it stays on + /// the blanket-rejection path. + #[test] + fn routing_picks_having_range_only_for_grouped_single_clause_having() { + let clause = hc( + having_aggregate::Function::Count, + "", + having_clause::Operator::GreaterThan, + Value::U64(2), + ); + + let grouped = GetDocumentsRequestV1 { + selects: select_count_star(), + group_by: vec![GROUP_PROPERTY.to_string()], + having: vec![clause.clone()], + limit: Some(10), + ..empty_v1_request() + }; + let label = validate_and_route_for_tests(&grouped, &[], PlatformVersion::latest()) + .expect("a grouped single-clause having is a supported shape"); + assert_eq!(label, "having_range"); + + // No group_by → the v0 blanket rejection still owns it. + let ungrouped = GetDocumentsRequestV1 { + group_by: Vec::new(), + ..grouped + }; + assert_not_yet_implemented( + validate_and_route_for_tests(&ungrouped, &[], PlatformVersion::latest()), + "HAVING clause", + ); + } +} + +mod having_trust_boundary { + //! The client trust boundary, exercised from the server side: a + //! grovedb-valid having proof is only an authenticated platform + //! result once drive-proof-verifier's [`verify_having_range_proof`] + //! wrapper binds its reconstructed root hash to the quorum-signed + //! app hash. These tests generate a real AVG having proof from a + //! real Drive, sign the canonical tenderdash precommit with a test + //! quorum key, and run the client wrapper end to end: the correctly + //! signed root verifies, and a commit over a different app hash, + //! tampered response metadata, or a wrong quorum key each fail — + //! so omitting or miswiring `verify_tenderdash_proof` turns a test + //! red. The suite lives here rather than in drive-proof-verifier + //! because generating proofs needs drive's server feature, which + //! the client crate must not enable even as a dev-dependency. + + use crate::platform_types::platform::Platform; + use crate::query::tests::{setup_platform, store_data_contract}; + use crate::rpc::core::MockCoreRPCLike; + use dapi_grpc::platform::v0::{Proof, ResponseMetadata}; + use dpp::block::block_info::BlockInfo; + use dpp::bls_signatures::{Bls12381G2Impl, SecretKey, SignatureSchemes}; + use dpp::dashcore::Network; + use dpp::data_contract::accessors::v0::DataContractV0Getters; + use dpp::data_contract::document_type::accessors::DocumentTypeV0Getters; + use dpp::data_contract::document_type::random_document::CreateRandomDocument; + use dpp::data_contract::TokenConfiguration; + use dpp::document::{Document, DocumentV0Setters}; + use dpp::platform_value::Value; + use dpp::prelude::{CoreBlockHeight, DataContract, Identifier}; + use dpp::tests::json_document::json_document_to_contract; + use dpp::version::PlatformVersion; + use drive::drive::Drive; + use drive::query::drive_document_having_query::drive_dispatcher::{ + DocumentHavingRequest, DocumentHavingResponse, + }; + use drive::query::drive_document_having_query::mode_detection::detect_having_mode; + use drive::query::drive_document_ranked_query::index_picker::find_ranked_index_for_axis; + use drive::query::having::{ + HavingAggregate, HavingAggregateFunction, HavingClause, HavingOperator, HavingRightOperand, + }; + use drive::query::projection::SelectProjection; + use drive::query::{DriveDocumentHavingQuery, RankedPaginationInputs}; + use drive::util::object_size_info::DocumentInfo::DocumentRefInfo; + use drive::util::object_size_info::{DocumentAndContractInfo, OwnedDocumentInfo}; + use drive_proof_verifier::{ + verify_having_range_proof, ContextProvider, ContextProviderError, + Error as ProofVerifierError, + }; + use std::collections::BTreeMap; + use std::sync::Arc; + use tenderdash_abci::proto::types::{CanonicalVote, SignedMsgType, StateId}; + use tenderdash_abci::signatures::{Hashable, Signable}; + + /// Shared with rs-drive's having suite — same cross-crate path + /// convention as `RESTAURANTS_CONTRACT_PATH` above. + const GRADES_RANKED_CONTRACT_PATH: &str = + "../rs-drive/tests/supporting_files/contract/grades/grades-ranked-contract.json"; + + const CHAIN_ID: &str = "test-having-chain"; + const HEIGHT: u64 = 4242; + const ROUND: u32 = 0; + const QUORUM_TYPE: u32 = 1; // LLMQ_50_60 + const CORE_LOCKED_HEIGHT: u32 = 1200; + const TIME_MS: u64 = 1_755_000_000_000; + + /// Provider that knows exactly one quorum key — the test one. + struct TestQuorumProvider { + pubkey: [u8; 48], + } + + impl ContextProvider for TestQuorumProvider { + fn get_data_contract( + &self, + _id: &Identifier, + _platform_version: &PlatformVersion, + ) -> Result>, ContextProviderError> { + Ok(None) + } + + fn get_token_configuration( + &self, + _token_id: &Identifier, + ) -> Result, ContextProviderError> { + Ok(None) + } + + fn get_quorum_public_key( + &self, + _quorum_type: u32, + _quorum_hash: [u8; 32], + _core_chain_locked_height: u32, + ) -> Result<[u8; 48], ContextProviderError> { + Ok(self.pubkey) + } + + fn get_platform_activation_height(&self) -> Result { + Ok(1) + } + } + + fn platform_version() -> &'static PlatformVersion { + PlatformVersion::latest() + } + + /// A deterministic, valid BLS scalar — no RNG dependency. + fn quorum_secret_key() -> SecretKey { + let mut bytes = [0u8; 32]; + bytes[31] = 42; + SecretKey::::from_be_bytes(&bytes) + .into_option() + .expect("a small nonzero scalar is a valid secret key") + } + + fn register_grades( + platform: &Platform, + platform_version: &PlatformVersion, + ) -> DataContract { + let contract = + json_document_to_contract(GRADES_RANKED_CONTRACT_PATH, false, platform_version) + .expect("expected to parse the ranked grades contract"); + store_data_contract(platform, &contract, platform_version); + contract + } + + /// A few grade documents so the axis secondary has content to + /// prove over: identity `[1; 32]` averages 75, identity `[2; 32]` + /// averages 90. + fn insert_grades(platform: &Platform, contract: &DataContract) { + let pv = platform_version(); + let document_type = contract + .document_type_for_name("grade") + .expect("grade doctype exists"); + let rows: [([u8; 32], i64); 4] = [ + ([1u8; 32], 70), + ([1u8; 32], 80), + ([2u8; 32], 85), + ([2u8; 32], 95), + ]; + for (i, (identity, grade)) in rows.iter().enumerate() { + let mut doc: Document = document_type + .random_document(Some(9000 + i as u64), pv) + .expect("random document"); + let mut props = BTreeMap::new(); + props.insert("identityId".to_string(), Value::Identifier(*identity)); + props.insert("grade".to_string(), Value::I64(*grade)); + doc.set_properties(props); + platform + .drive + .add_document_for_contract( + DocumentAndContractInfo { + owned_document_info: OwnedDocumentInfo { + document_info: DocumentRefInfo((&doc, None)), + owner_id: None, + }, + contract, + document_type, + }, + false, + BlockInfo::default(), + true, + None, + pv, + None, + ) + .expect("expected to insert a grade document"); + } + } + + /// `AVG(grade) > 80 LIMIT 10` — matches identity `[2; 32]` + /// (average 90) and excludes identity `[1; 32]` (average 75). + fn having_clause() -> HavingClause { + HavingClause { + aggregate: HavingAggregate { + function: HavingAggregateFunction::Avg, + field: "grade".to_string(), + }, + operator: HavingOperator::GreaterThan, + right: HavingRightOperand::Value(Value::U64(80)), + } + } + + fn client_side_query(contract: &DataContract) -> DriveDocumentHavingQuery<'_> { + let group_by = vec!["identityId".to_string()]; + let having = vec![having_clause()]; + let mode = detect_having_mode( + &SelectProjection::avg("grade"), + &group_by, + &having, + &[], + &[], + RankedPaginationInputs { + limit: Some(10), + offset: None, + has_start_at: false, + }, + platform_version(), + ) + .expect("the case is well-formed"); + let index = find_ranked_index_for_axis( + contract + .document_types() + .get("grade") + .expect("grade doctype exists") + .indexes(), + &mode.group_by_property, + mode.bounds.axis(), + &mode.aggregate_field, + ) + .expect("the fixture declares the avg axis"); + DriveDocumentHavingQuery { + document_type: contract + .document_type_for_name("grade") + .expect("grade doctype exists"), + contract_id: contract.id_ref().to_buffer(), + document_type_name: "grade".to_string(), + index, + bounds: mode.bounds, + descending: mode.descending, + limit: mode.limit, + } + } + + /// Prove the having request against the live Drive and return + /// `(grovedb proof bytes, live root hash)`. + fn prove(drive: &Drive, contract: &DataContract) -> (Vec, [u8; 32]) { + let group_by = vec!["identityId".to_string()]; + let having = vec![having_clause()]; + let response = drive + .execute_document_having_request( + DocumentHavingRequest { + contract, + document_type: contract + .document_type_for_name("grade") + .expect("grade doctype exists"), + group_by: &group_by, + select: SelectProjection::avg("grade"), + having: &having, + order_by: &[], + where_clauses: &[], + limit: Some(10), + offset: None, + has_start_at: false, + prove: true, + }, + None, + platform_version(), + ) + .expect("the prove request must execute"); + let proof_bytes = match response { + DocumentHavingResponse::Proof(proof) => proof, + DocumentHavingResponse::Entries(_) => panic!("expected a proof, got entries"), + }; + let root_hash = drive + .grove + .root_hash(None, &platform_version().drive.grove_version) + .unwrap() + .expect("root hash must be readable"); + (proof_bytes, root_hash) + } + + fn metadata() -> ResponseMetadata { + ResponseMetadata { + height: HEIGHT, + core_chain_locked_height: CORE_LOCKED_HEIGHT, + epoch: 0, + time_ms: TIME_MS, + protocol_version: platform_version().protocol_version, + chain_id: CHAIN_ID.to_string(), + } + } + + /// Sign a tenderdash precommit whose state id carries `app_hash` — + /// the same canonical construction `verify_tenderdash_proof` + /// rebuilds on the verify side. + fn signed_proof( + grovedb_proof: Vec, + app_hash: &[u8; 32], + mtd: &ResponseMetadata, + secret_key: &SecretKey, + quorum_hash: [u8; 32], + ) -> Proof { + let block_id_hash = [7u8; 32].to_vec(); + let state_id = StateId { + app_version: mtd.protocol_version as u64, + core_chain_locked_height: mtd.core_chain_locked_height, + time: mtd.time_ms, + app_hash: app_hash.to_vec(), + height: mtd.height, + }; + let state_id_hash = state_id + .calculate_msg_hash(&mtd.chain_id, mtd.height as i64, ROUND as i32) + .expect("state id hash"); + let commit = CanonicalVote { + r#type: SignedMsgType::Precommit.into(), + block_id: block_id_hash.clone(), + chain_id: mtd.chain_id.clone(), + height: mtd.height as i64, + round: ROUND as i64, + state_id: state_id_hash, + }; + let sign_digest = commit + .calculate_sign_hash( + &mtd.chain_id, + QUORUM_TYPE.try_into().expect("valid quorum type"), + &quorum_hash, + mtd.height as i64, + ROUND as i32, + ) + .expect("sign digest"); + let signature = secret_key + .sign(SignatureSchemes::Basic, &sign_digest) + .expect("signing with a valid key succeeds") + .as_raw_value() + .to_compressed() + .to_vec(); + Proof { + grovedb_proof, + quorum_hash: quorum_hash.to_vec(), + signature, + round: ROUND, + block_id_hash, + quorum_type: QUORUM_TYPE, + } + } + + /// The full composition succeeds end to end: merk verification + /// reconstructs the root, the tenderdash commit over that root + /// verifies against the quorum key, and the verified entries are + /// exactly the matching groups. + #[test] + fn a_correctly_signed_root_verifies_and_returns_the_matches() { + let (platform, _state, _version) = setup_platform(None, Network::Testnet, None); + let contract = register_grades(&platform, platform_version()); + insert_grades(&platform, &contract); + let (grovedb_proof, root_hash) = prove(&platform.drive, &contract); + let secret_key = quorum_secret_key(); + let quorum_hash = [3u8; 32]; + let mtd = metadata(); + let proof = signed_proof(grovedb_proof, &root_hash, &mtd, &secret_key, quorum_hash); + let provider = TestQuorumProvider { + pubkey: secret_key.public_key().0.to_compressed(), + }; + + let query = client_side_query(&contract); + let (verified_root, entries) = + verify_having_range_proof(&query, &proof, &mtd, platform_version(), &provider) + .expect("a correctly signed root must verify"); + + assert_eq!(verified_root, root_hash); + assert_eq!( + entries.iter().map(|e| e.key.clone()).collect::>(), + vec![[2u8; 32].to_vec()], + "only the identity averaging 90 clears the > 80 bound" + ); + } + + /// A commit signed over a *different* app hash must not verify: + /// the node's grovedb proof reconstructs the true root, and the + /// tenderdash binding is what catches the mismatch. + #[test] + fn a_commit_over_a_different_app_hash_is_rejected() { + let (platform, _state, _version) = setup_platform(None, Network::Testnet, None); + let contract = register_grades(&platform, platform_version()); + insert_grades(&platform, &contract); + let (grovedb_proof, _root_hash) = prove(&platform.drive, &contract); + let secret_key = quorum_secret_key(); + let quorum_hash = [3u8; 32]; + let mtd = metadata(); + let wrong_app_hash = [0xAA; 32]; + let proof = signed_proof( + grovedb_proof, + &wrong_app_hash, + &mtd, + &secret_key, + quorum_hash, + ); + let provider = TestQuorumProvider { + pubkey: secret_key.public_key().0.to_compressed(), + }; + + let query = client_side_query(&contract); + let error = verify_having_range_proof(&query, &proof, &mtd, platform_version(), &provider) + .expect_err("a commit over a different app hash must be rejected"); + assert!( + matches!(error, ProofVerifierError::InvalidSignature { .. }), + "the rejection must be the signature binding, got: {error:?}" + ); + } + + /// Tampered response metadata changes the canonical state id, so a + /// signature over the honest metadata stops verifying. + #[test] + fn tampered_metadata_is_rejected() { + let (platform, _state, _version) = setup_platform(None, Network::Testnet, None); + let contract = register_grades(&platform, platform_version()); + insert_grades(&platform, &contract); + let (grovedb_proof, root_hash) = prove(&platform.drive, &contract); + let secret_key = quorum_secret_key(); + let quorum_hash = [3u8; 32]; + let mtd = metadata(); + let proof = signed_proof(grovedb_proof, &root_hash, &mtd, &secret_key, quorum_hash); + let provider = TestQuorumProvider { + pubkey: secret_key.public_key().0.to_compressed(), + }; + + let mut tampered = mtd; + tampered.height += 1; + + let query = client_side_query(&contract); + let error = + verify_having_range_proof(&query, &proof, &tampered, platform_version(), &provider) + .expect_err("tampered metadata must be rejected"); + assert!( + matches!(error, ProofVerifierError::InvalidSignature { .. }), + "the rejection must be the signature binding, got: {error:?}" + ); + } + + /// A provider vending a different quorum key models a signer + /// outside the expected quorum: the commit must not verify. + #[test] + fn a_wrong_quorum_key_is_rejected() { + let (platform, _state, _version) = setup_platform(None, Network::Testnet, None); + let contract = register_grades(&platform, platform_version()); + insert_grades(&platform, &contract); + let (grovedb_proof, root_hash) = prove(&platform.drive, &contract); + let secret_key = quorum_secret_key(); + let quorum_hash = [3u8; 32]; + let mtd = metadata(); + let proof = signed_proof(grovedb_proof, &root_hash, &mtd, &secret_key, quorum_hash); + + let mut other_bytes = [0u8; 32]; + other_bytes[31] = 43; + let other_key = SecretKey::::from_be_bytes(&other_bytes) + .into_option() + .expect("valid scalar"); + let provider = TestQuorumProvider { + pubkey: other_key.public_key().0.to_compressed(), + }; + + let query = client_side_query(&contract); + let error = verify_having_range_proof(&query, &proof, &mtd, platform_version(), &provider) + .expect_err("a commit signed outside the expected quorum must be rejected"); + assert!( + matches!(error, ProofVerifierError::InvalidSignature { .. }), + "the rejection must be the signature binding, got: {error:?}" + ); + } +} diff --git a/packages/rs-drive-proof-verifier/src/lib.rs b/packages/rs-drive-proof-verifier/src/lib.rs index 0a5bf872d42..f91801de6e1 100644 --- a/packages/rs-drive-proof-verifier/src/lib.rs +++ b/packages/rs-drive-proof-verifier/src/lib.rs @@ -14,6 +14,14 @@ pub use proof::document_count::{ verify_distinct_count_proof, verify_point_lookup_count_proof, verify_primary_key_count_tree_proof, DocumentCount, }; +/// Verified having-range (`GROUP BY … HAVING +/// LIMIT n`) result types. `DocumentHavingEntries` carries one entry +/// per matching group **in axis order**; +/// [`verify_having_range_proof`] is the tenderdash-composition wrapper +/// that binds the proof's reconstructed root hash to the signed app +/// hash and returns the verified entry list — including its +/// completeness: an in-range group the node omitted fails verification. +pub use proof::document_having::{verify_having_range_proof, DocumentHavingEntries}; /// Verified ranked (`GROUP BY … ORDER BY LIMIT n /// [OFFSET m]`) result types. `DocumentRankedEntries` carries one entry /// per returned group **in ranking order**, plus the `starting_rank` diff --git a/packages/rs-drive-proof-verifier/src/proof.rs b/packages/rs-drive-proof-verifier/src/proof.rs index eb47bdab25c..ffcd66171da 100644 --- a/packages/rs-drive-proof-verifier/src/proof.rs +++ b/packages/rs-drive-proof-verifier/src/proof.rs @@ -4,6 +4,11 @@ /// `AggregateCountAndSumOnRange` primitive. pub mod document_average; pub mod document_count; +/// Verified having-range (`GROUP BY … HAVING +/// LIMIT n`) result. One entry per matching group, in axis order, read +/// as a value-bounded range of an indexed tree's per-axis secondary +/// (grovedb PR 657); see the file's docs. +pub mod document_having; /// Verified ranked (`GROUP BY … ORDER BY LIMIT n /// [OFFSET m]`) result. One entry per returned group, in ranking order, /// plus the attested rank the page starts at, read from an indexed diff --git a/packages/rs-drive-proof-verifier/src/proof/document_having.rs b/packages/rs-drive-proof-verifier/src/proof/document_having.rs new file mode 100644 index 00000000000..ea6069a6c10 --- /dev/null +++ b/packages/rs-drive-proof-verifier/src/proof/document_having.rs @@ -0,0 +1,341 @@ +//! Verified **having-range** +//! (`GROUP BY … HAVING LIMIT n`) document +//! results. +//! +//! A having-range query answers "which groups' aggregate falls inside a +//! value bound?" — `SELECT COUNT(*) GROUP BY hashtag HAVING $count > 100 +//! LIMIT 100`. The answer is a value-bounded range read of the same +//! per-axis *secondary* Merk the ranked query walks, so it costs +//! `O(log n + k)` and comes with a proof that commits to exactly the +//! returned `(aggregate, group key)` pairs **and their completeness**: +//! the Merk range proof commits its boundaries, so an in-range group the +//! node omitted fails verification. +//! +//! This module holds the client-facing result type +//! ([`DocumentHavingEntries`]), the tenderdash-composition wrapper +//! around rs-drive's merk-level verifier +//! ([`verify_having_range_proof`]), and the decoder for the unproven +//! wire payload ([`DocumentHavingEntries::from_unproved_response`]) — +//! which rides the same `ResultData.ranked` variant the ranked surface +//! uses, since a having page is the same "group key + aggregate value" +//! entry list. +//! +//! Per-shape routing (which index covers the axis, which bounds the +//! clause translates to) lives in rs-sdk's `having_proof_helpers`, +//! exactly as the ranked equivalents live in `ranked_proof_helpers` — +//! it needs the data contract, which this crate does not carry. + +use crate::error::MapGroveDbError; +use crate::proof::document_ranked::{ranked_entry_from_proto, result_variant_name}; +use crate::verify::verify_tenderdash_proof; +use crate::{ContextProvider, Error, FromProof}; +use dapi_grpc::platform::v0::get_documents_response::get_documents_response_v1::{ + result_data, ResultData, +}; +use dapi_grpc::platform::v0::get_documents_response::{ + get_documents_response_v1, Version as ResponseVersion, +}; +use dapi_grpc::platform::v0::{GetDocumentsResponse, Proof, ResponseMetadata}; +use dpp::dashcore::Network; +use dpp::version::PlatformVersion; +use drive::query::{DriveDocumentHavingQuery, DriveDocumentQuery, RankedEntry}; +use drive::verify::RootHash; + +/// One page of a `GROUP BY … HAVING LIMIT n` +/// query: the groups whose aggregate falls inside the bound. +/// +/// **Entry order is axis order in the walk direction** — ascending by +/// default, descending when the request ordered by the aggregate +/// descending. Callers must not re-sort; ties (groups with equal +/// aggregates) come back in group-key order in the direction of the +/// walk, same as on the ranked surface. +/// +/// Fewer than `n` entries means fewer groups matched — not an error. +/// **Exactly `n` entries may mean the match set was cut at the limit**; +/// nothing in the page marks the cut. Tightening the bound past the +/// last aggregate value seen continues past *distinct* values only — a +/// cut inside a tie (several groups sharing the boundary aggregate) +/// cannot be continued, so size the limit above the widest expected +/// tie. +/// +/// Entry semantics ([`RankedEntry`]) are identical to the ranked +/// surface's, including the fixed-point average scaling and the +/// exact-on-the-proved-path-only caveat. +#[derive(Debug, Clone, PartialEq, Eq, Default)] +pub struct DocumentHavingEntries { + /// The matching groups, in axis order in the walk direction. + pub entries: Vec, +} + +impl DocumentHavingEntries { + /// Build a [`DocumentHavingEntries`] from the verifier-side entry + /// list — the shape rs-drive's merk-level verifier returns. + pub fn from_verified(entries: Vec) -> Self { + DocumentHavingEntries { entries } + } + + /// Decode the **unproven** having-range payload of a `getDocuments` + /// response — the `ResultData.ranked` variant a node returns for a + /// having-range request sent with `prove = false`. (The wire reuses + /// the ranked entries message; a having page leaves its `skipped` + /// field unset, and this decoder ignores it either way, because a + /// value-bounded page has no rank base for it to describe.) + /// + /// Order is preserved verbatim. This is a plain wire decode with + /// **no cryptographic guarantee whatsoever** — and unlike the ranked + /// surface the missing guarantee here includes *completeness*: an + /// unproven page is free to omit matching groups, which for a + /// spam-resistance query is precisely the interesting attack. Prefer + /// [`verify_having_range_proof`] (via rs-sdk's + /// `DocumentHavingEntries::fetch`) unless you deliberately trust the + /// node. + /// + /// # Errors + /// + /// - [`Error::EmptyVersion`] when the response carries no version. + /// - [`Error::ResponseDecodeError`] when the response is a V0 + /// response, carries a proof rather than data, carries a + /// non-ranked `ResultData` variant, or an entry's `value` oneof is + /// unset / out of domain. + pub fn from_unproved_response( + response: &GetDocumentsResponse, + ) -> Result<(Self, ResponseMetadata), Error> { + let version = response.version.as_ref().ok_or(Error::EmptyVersion)?; + let ResponseVersion::V1(v1) = version else { + return Err(Error::ResponseDecodeError { + error: "having-range results are a V1-only response shape; got a V0 \ + getDocuments response. Having-range queries require protocol \ + version 14+." + .to_string(), + }); + }; + let metadata = v1.metadata.clone().ok_or(Error::EmptyResponseMetadata)?; + let entries = match v1.result.as_ref() { + Some(get_documents_response_v1::Result::Data(ResultData { + variant: Some(result_data::Variant::Ranked(ranked)), + })) => ranked + .entries + .iter() + .map(ranked_entry_from_proto) + .collect::, _>>()?, + Some(get_documents_response_v1::Result::Proof(_)) => { + return Err(Error::ResponseDecodeError { + error: "the response carries a proof, not unproven having-range entries; \ + verify it with `verify_having_range_proof` instead of decoding it" + .to_string(), + }); + } + other => { + return Err(Error::ResponseDecodeError { + error: format!( + "expected a `ResultData.ranked` payload for a having-range request, \ + got {}. A response on another variant means the node routed \ + the request to a different executor — check that the request \ + carries a `group_by` and exactly one `having` clause bounding the \ + single `select`'s aggregate.", + result_variant_name(other) + ), + }); + } + }; + Ok((DocumentHavingEntries { entries }, metadata)) + } +} + +/// Verify a grovedb indexed-axis range proof **and the surrounding +/// tenderdash commit**, returning the reconstructed root hash and the +/// matching groups it commits to. +/// +/// Thin tenderdash-composition wrapper over +/// [`DriveDocumentHavingQuery::verify_having_range_proof`] in rs-drive +/// (which does the merk-level verification). Both sides derive the +/// proved subtree from the same +/// `DriveDocumentHavingQuery::indexed_property_name_tree_path` and the +/// secondary query from the same `AxisRangeBounds::merk_query`, so +/// prover and verifier cannot drift on *which bound over which tree* is +/// being checked, and grovedb re-checks the echoed query and limit — a +/// proof of one bound does not verify as another. +/// +/// ## The root hash is the whole point +/// +/// Same as on the ranked surface: the merk-level verifier returning +/// `Ok` is not by itself evidence of anything — the binding to the +/// quorum-signed app hash in [`verify_tenderdash_proof`] is what makes +/// the entries (and their completeness) attested facts. This function +/// exists so that composition can never be skipped by accident. +pub fn verify_having_range_proof( + query: &DriveDocumentHavingQuery, + proof: &Proof, + mtd: &ResponseMetadata, + platform_version: &PlatformVersion, + provider: &dyn ContextProvider, +) -> Result<(RootHash, Vec), Error> { + let (root_hash, entries) = query + .verify_having_range_proof(&proof.grovedb_proof, platform_version) + .map_drive_error(proof, mtd)?; + + verify_tenderdash_proof(proof, mtd, &root_hash, provider)?; + + Ok((root_hash, entries)) +} + +/// Reject the generic [`FromProof`] entry point for +/// [`DocumentHavingEntries`] — same guard rail, same rationale as the +/// [`crate::DocumentRankedEntries`] blanket impl: the generic +/// `FromProof>` path carries neither the +/// bounds nor the covering index, so it errors out explicitly rather +/// than verifying the wrong thing. +impl<'dq, Q> FromProof for DocumentHavingEntries +where + Q: TryInto> + Clone + 'dq, + Q::Error: std::fmt::Display, +{ + type Request = Q; + type Response = GetDocumentsResponse; + + fn maybe_from_proof_with_metadata<'a, I: Into, O: Into>( + _request: I, + _response: O, + _network: Network, + _platform_version: &PlatformVersion, + _provider: &'a dyn ContextProvider, + ) -> Result<(Option, ResponseMetadata, Proof), Error> + where + Self: 'a, + { + Err(Error::RequestError { + error: "DocumentHavingEntries can't be verified via the generic FromProof path; \ + call DocumentHavingEntries::fetch on a DocumentQuery carrying \ + .with_select(), .with_group_by(), \ + .with_having() and \ + .with_limit(n), which resolves the bounds and the covering index from \ + the data contract" + .to_string(), + }) + } +} + +#[cfg(test)] +mod tests { + //! Offline tests for the unproven decode and the response-shape + //! rejections. Proof verification is exercised end-to-end by + //! rs-drive's `drive_document_having_query::tests` (prover and + //! merk-level verifier against a real Drive), rs-drive-abci's + //! `having_range_tests` (wire encoding of the same values), and + //! rs-drive-abci's `having_trust_boundary` suite, which runs this + //! crate's [`verify_having_range_proof`] wrapper — including the + //! tenderdash signature binding — against server-generated proofs. + //! The tenderdash-composition tests live on the server side so this + //! client crate keeps building drive with `verify` only. + use super::*; + use dapi_grpc::platform::v0::get_documents_response::get_documents_response_v1::{ + ranked_entry, Documents, RankedEntries, RankedEntry as ProtoRankedEntry, + }; + use dapi_grpc::platform::v0::get_documents_response::GetDocumentsResponseV1; + use drive::query::RankedEntryValue; + + fn count_entry(key: &str, count: u64) -> ProtoRankedEntry { + ProtoRankedEntry { + key: key.as_bytes().to_vec(), + value: Some(ranked_entry::Value::Count(count)), + } + } + + fn response_with(result: get_documents_response_v1::Result) -> GetDocumentsResponse { + GetDocumentsResponse { + version: Some(ResponseVersion::V1(GetDocumentsResponseV1 { + result: Some(result), + metadata: Some(ResponseMetadata { + height: 42, + ..Default::default() + }), + })), + } + } + + fn having_response( + entries: Vec, + skipped: Option, + ) -> GetDocumentsResponse { + response_with(get_documents_response_v1::Result::Data(ResultData { + variant: Some(result_data::Variant::Ranked(RankedEntries { + entries, + skipped, + })), + })) + } + + /// The headline decode: `HAVING $count > 100`-shaped entries come + /// back in axis order, untouched, with `skipped` (unset on a + /// having page) ignored. + #[test] + fn decodes_entries_preserving_axis_order() { + let response = having_response( + vec![count_entry("dash", 101), count_entry("evo", 250)], + None, + ); + let (decoded, metadata) = DocumentHavingEntries::from_unproved_response(&response) + .expect("a well-formed having payload decodes"); + assert_eq!(metadata.height, 42); + assert_eq!( + decoded.entries.iter().map(|e| e.value).collect::>(), + vec![RankedEntryValue::Count(101), RankedEntryValue::Count(250)] + ); + } + + /// A stray `skipped` from a non-conforming node is ignored, not a + /// decode failure: the field cannot describe anything on a + /// value-bounded page, and failing on it would break against a + /// node that reused its ranked encoder wholesale. + #[test] + fn a_stray_skipped_field_is_ignored() { + let response = having_response(vec![count_entry("dash", 101)], Some(7)); + let (decoded, _) = DocumentHavingEntries::from_unproved_response(&response) + .expect("a stray skipped is not a decode failure"); + assert_eq!(decoded.entries.len(), 1); + } + + /// No groups matching the bound is a legitimate answer. + #[test] + fn decodes_an_empty_match_set() { + let (decoded, _) = + DocumentHavingEntries::from_unproved_response(&having_response(vec![], None)) + .expect("an empty match set is well-formed"); + assert!(decoded.entries.is_empty()); + } + + /// Same caller-mistake guard as the ranked decoder: a proof must + /// be verified, not decoded. + #[test] + fn rejects_a_proof_response() { + let response = response_with(get_documents_response_v1::Result::Proof(Proof::default())); + let err = DocumentHavingEntries::from_unproved_response(&response) + .expect_err("a proof is not an unproven having payload"); + assert!(format!("{err}").contains("verify_having_range_proof")); + } + + /// A response on another variant means the node routed the request + /// somewhere else entirely. + #[test] + fn rejects_a_non_ranked_result_variant() { + let response = response_with(get_documents_response_v1::Result::Data(ResultData { + variant: Some(result_data::Variant::Documents(Documents { + documents: Vec::new(), + })), + })); + let err = DocumentHavingEntries::from_unproved_response(&response) + .expect_err("a documents payload is not a having one"); + assert!(format!("{err}").contains("ResultData.ranked")); + } + + /// V0 predates the SQL-shaped surface entirely. + #[test] + fn rejects_a_v0_response() { + let response = GetDocumentsResponse { + version: Some(ResponseVersion::V0(Default::default())), + }; + let err = DocumentHavingEntries::from_unproved_response(&response) + .expect_err("V0 has no having shape"); + assert!(format!("{err}").contains("V1-only")); + } +} diff --git a/packages/rs-drive-proof-verifier/src/proof/document_ranked.rs b/packages/rs-drive-proof-verifier/src/proof/document_ranked.rs index 6239eca48e9..63bd8d1e01e 100644 --- a/packages/rs-drive-proof-verifier/src/proof/document_ranked.rs +++ b/packages/rs-drive-proof-verifier/src/proof/document_ranked.rs @@ -188,10 +188,11 @@ impl DocumentRankedEntries { return Err(Error::ResponseDecodeError { error: format!( "expected a `ResultData.ranked` payload for a ranked request, got \ - {other:?}. A response on another variant means the node routed the \ + {}. A response on another variant means the node routed the \ request to a different executor — check that the request carries a \ `group_by` and a single `order_by` naming the single `select`'s \ - aggregate (`$count` for `COUNT(*)`)." + aggregate (`$count` for `COUNT(*)`).", + result_variant_name(other) ), }); } @@ -206,6 +207,28 @@ impl DocumentRankedEntries { } } +/// The received-but-unexpected shape of a `getDocuments` V1 result, by +/// **name only** — never the payload. Interpolating the payload into an +/// error would make the message (and any log line carrying it) grow +/// with an untrusted response, and could copy returned document bytes +/// into logs. Shared by the ranked and having-range decoders. +pub(crate) fn result_variant_name( + result: Option<&get_documents_response_v1::Result>, +) -> &'static str { + match result { + None => "an absent result", + Some(get_documents_response_v1::Result::Proof(_)) => "a proof", + Some(get_documents_response_v1::Result::Data(ResultData { variant })) => match variant { + None => "a ResultData with no variant", + Some(result_data::Variant::Documents(_)) => "a ResultData.documents payload", + Some(result_data::Variant::Counts(_)) => "a ResultData.counts payload", + Some(result_data::Variant::Sums(_)) => "a ResultData.sums payload", + Some(result_data::Variant::Averages(_)) => "a ResultData.averages payload", + Some(result_data::Variant::Ranked(_)) => "a ResultData.ranked payload", + }, + } +} + /// Decode one wire [`ProtoRankedEntry`] into rs-drive's /// [`RankedEntry`]. /// @@ -224,7 +247,7 @@ impl DocumentRankedEntries { /// out-of-range double into `i128::MIN`/`MAX`. Every legitimate value /// fits comfortably, since `|sum| ≤ i64::MAX` bounds the true fixed /// point at `i64::MAX * 10^19 ≈ 9.2e37 < i128::MAX`. -fn ranked_entry_from_proto(entry: &ProtoRankedEntry) -> Result { +pub(crate) fn ranked_entry_from_proto(entry: &ProtoRankedEntry) -> Result { let value = match entry.value.as_ref() { Some(ranked_entry::Value::Count(count)) => RankedEntryValue::Count(*count), Some(ranked_entry::Value::Sum(sum)) => RankedEntryValue::Sum(*sum), diff --git a/packages/rs-drive/src/query/drive_document_having_query/drive_dispatcher.rs b/packages/rs-drive/src/query/drive_document_having_query/drive_dispatcher.rs new file mode 100644 index 00000000000..8c5b35da0c6 --- /dev/null +++ b/packages/rs-drive/src/query/drive_document_having_query/drive_dispatcher.rs @@ -0,0 +1,147 @@ +//! [`DocumentHavingRequest`] / [`DocumentHavingResponse`] and the +//! having-range dispatcher on `impl Drive` — the ABI drive-abci's +//! routing layer names. + +use super::super::drive_document_ranked_query::{RankedEntry, RankedPaginationInputs}; +use super::mode_detection::detect_having_mode; +use crate::drive::Drive; +use crate::error::Error; +use crate::query::having::HavingClause; +use crate::query::projection::SelectProjection; +use crate::query::{OrderClause, WhereClause}; +use dpp::data_contract::accessors::v0::DataContractV0Getters; +use dpp::data_contract::document_type::accessors::DocumentTypeV0Getters; +use dpp::data_contract::document_type::DocumentTypeRef; +use dpp::data_contract::DataContract; +use dpp::version::PlatformVersion; +use grovedb::TransactionArg; + +/// All inputs required by [`Drive::execute_document_having_request`]. +/// Built by the gRPC handler from a `GetDocumentsRequestV1` after +/// wire-decoding + contract lookup — the same construction pattern as +/// [`super::super::drive_document_ranked_query::DocumentRankedRequest`]. +/// +/// `where_clauses`, `offset` and `start_at` are carried even though a +/// having-range request must leave all of them empty: drive owns the +/// rejection, so the contract is enforced identically no matter which +/// upstream path built the request. See +/// [`super::mode_detection::detect_having_mode_v0`] for why each is +/// refused rather than ignored. +pub struct DocumentHavingRequest<'a> { + /// Live contract (already loaded by the handler). + pub contract: &'a DataContract, + /// Resolved document type within `contract`. + pub document_type: DocumentTypeRef<'a>, + /// The single `GROUP BY` property. Must be the ranked index's only + /// property. + pub group_by: &'a [String], + /// The projection whose aggregate the `having` clause bounds: + /// `COUNT(*)`, `SUM(field)` or `AVG(field)`. + pub select: SelectProjection, + /// The `HAVING` clauses. Exactly one, bounding the selected + /// aggregate. + pub having: &'a [HavingClause], + /// The `ORDER BY` clauses. Empty (ascending default) or exactly + /// one, naming the selected aggregate. + pub order_by: &'a [OrderClause], + /// Structured `where` clauses. Must be empty. + pub where_clauses: &'a [WhereClause], + /// Request `limit`. **Required**; `1 ..= MAX_HAVING_LIMIT`. + pub limit: Option, + /// Request `offset`. Must be `None` — the range walk has no skip. + pub offset: Option, + /// Whether the request carried a `start_at` / `start_after` cursor. + /// Must be `false`. + pub has_start_at: bool, + /// Whether to produce a proof instead of materializing entries. + pub prove: bool, +} + +/// Output shape of [`Drive::execute_document_having_request`]. +/// +/// - `Entries` — the matching groups **in axis order in the walk +/// direction**; the abci handler maps this straight onto the wire's +/// ranked-entries shape (with no rank base) without re-sorting. +/// - `Proof(Vec)` — grovedb indexed-axis range proof bytes the +/// client verifies with +/// [`DriveDocumentHavingQuery::verify_having_range_proof`](crate::query::DriveDocumentHavingQuery::verify_having_range_proof), +/// which recovers the same entry list. +#[derive(Debug, Clone)] +pub enum DocumentHavingResponse { + /// The groups whose aggregate falls inside the bound, cut at the + /// request's limit. + Entries(Vec), + /// Grovedb indexed-axis range proof bytes. + Proof(Vec), +} + +impl Drive { + /// Single entry point for a having-range document request. + /// + /// 1. [`detect_having_mode`] validates the request shape and + /// resolves the `(bounds, descending, limit, group property, + /// aggregate field)` tuple. + /// 2. The matching executor picks the covering ranked index and runs + /// the read or the proof. + /// 3. The result is wrapped in [`DocumentHavingResponse`]. + /// + /// Errors: + /// - Request-shape failures (wrong `group_by` arity, a clause on an + /// aggregate the select does not project, an untranslatable + /// operator, a missing or out-of-range `limit`, a `where`, an + /// `offset`) come back as `Error::Query(QuerySyntaxError::*)` — + /// see [`super::mode_detection::detect_having_mode_v0`] for the + /// full grammar. + /// - "No index declares this axis" comes back as + /// `Error::Query(QuerySyntaxError::WhereClauseOnNonIndexedProperty)` + /// naming the missing contract keyword. + /// - Everything else (grovedb, versioning) surfaces as its native + /// `Error` variant. + pub fn execute_document_having_request( + &self, + request: DocumentHavingRequest, + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result { + let mode = detect_having_mode( + &request.select, + request.group_by, + request.having, + request.order_by, + request.where_clauses, + RankedPaginationInputs { + limit: request.limit, + offset: request.offset, + has_start_at: request.has_start_at, + }, + platform_version, + )?; + + let contract_id = request.contract.id_ref().to_buffer(); + let document_type_name = request.document_type.name().to_string(); + + if request.prove { + Ok(DocumentHavingResponse::Proof( + self.execute_document_having_range_proof( + contract_id, + request.document_type, + document_type_name, + &mode, + transaction, + platform_version, + )?, + )) + } else { + Ok(DocumentHavingResponse::Entries( + self.execute_document_having_range_no_proof( + contract_id, + request.document_type, + document_type_name, + &mode, + transaction, + platform_version, + )?, + )) + } + } +} diff --git a/packages/rs-drive/src/query/drive_document_having_query/execute_range.rs b/packages/rs-drive/src/query/drive_document_having_query/execute_range.rs new file mode 100644 index 00000000000..c9188d91f93 --- /dev/null +++ b/packages/rs-drive/src/query/drive_document_having_query/execute_range.rs @@ -0,0 +1,176 @@ +//! The two having-range executors on [`DriveDocumentHavingQuery`]: a +//! direct value-bounded read of the axis secondary, and generation of +//! the equivalent proof. +//! +//! Both are thin — all of the work happens inside grovedb, which seeks +//! straight to the encoded bounds in the pre-sorted secondary Merk. No +//! value trees are opened, no documents are materialized, and the cost +//! is `O(log n + k)` in the number of *matching* groups returned, never +//! in the total group population. +//! +//! Whole module is gated `feature = "server"` via the parent's +//! `pub mod execute_range;` declaration. + +use super::super::drive_document_ranked_query::{RankedAxis, RankedEntry, RankedEntryValue}; +use super::{AxisRangeBounds, DriveDocumentHavingQuery}; +use crate::drive::Drive; +use crate::error::drive::DriveError; +use crate::error::Error; +use dpp::version::PlatformVersion; +use grovedb::TransactionArg; +use grovedb_costs::CostContext; + +impl DriveDocumentHavingQuery<'_> { + /// Read the matching groups directly from the axis secondary: every + /// group whose aggregate falls inside the bounds, up to `limit`, in + /// axis order in the walk direction. + /// + /// Fewer than `limit` entries is normal (fewer groups match) and is + /// not an error; exactly `limit` entries may mean the match set was + /// cut. A missing path *is* an error rather than an empty result, + /// for the same reason as on the ranked surface: the indexed + /// property-name tree is created at contract registration, so its + /// absence means the contract-level state is not what the request + /// claims. (An index with no documents has the tree, with an empty + /// secondary, and yields an empty entry list.) + pub fn execute_range_no_proof( + &self, + drive: &Drive, + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result, Error> { + let grove_version = &platform_version.drive.grove_version; + let path = self.indexed_property_name_tree_path()?; + let path_refs: Vec<&[u8]> = path.iter().map(|segment| segment.as_slice()).collect(); + + // Costs are destructured away rather than `.unwrap()`-ed, same + // as the ranked executors: `CostContext::unwrap` is infallible + // but reads like a panicking unwrap at the call site. + let entries = match self.bounds { + AxisRangeBounds::Count { lo, hi } => { + let CostContext { value, cost: _ } = drive.grove.indexed_count_range( + path_refs.as_slice(), + lo, + hi, + self.descending, + self.limit, + transaction, + grove_version, + ); + value + .map_err(|e| Error::GroveDB(Box::new(e)))? + .into_iter() + .map(|(count, key)| RankedEntry { + key, + value: RankedEntryValue::Count(count), + }) + .collect::>() + } + AxisRangeBounds::Sum { lo, hi } => { + let CostContext { value, cost: _ } = drive.grove.indexed_sum_range( + path_refs.as_slice(), + lo, + hi, + self.descending, + self.limit, + transaction, + grove_version, + ); + value + .map_err(|e| Error::GroveDB(Box::new(e)))? + .into_iter() + .map(|(sum, key)| RankedEntry { + key, + value: RankedEntryValue::Sum(sum), + }) + .collect::>() + } + AxisRangeBounds::Avg { lo, hi } => { + let CostContext { value, cost: _ } = drive.grove.indexed_avg_range( + path_refs.as_slice(), + lo, + hi, + self.descending, + self.limit, + transaction, + grove_version, + ); + value + .map_err(|e| Error::GroveDB(Box::new(e)))? + .into_iter() + .map(|(avg, key)| RankedEntry { + key, + value: RankedEntryValue::AvgFixedPoint(avg), + }) + .collect::>() + } + }; + + // The limit is the contract with the caller, and on the prove + // path it is re-checked inside the proof envelope. Asserting it + // here keeps the no-proof and prove responses shape-identical. + if entries.len() > self.limit as usize { + return Err(Error::Drive(DriveError::CorruptedDriveState(format!( + "having {:?} range read returned {} entries for limit = {}", + self.bounds.axis(), + entries.len(), + self.limit + )))); + } + Ok(entries) + } + + /// Generate the grovedb indexed-axis range proof for this query. + /// + /// The envelope commits the in-range secondary entries, the + /// primary's root hash, the sibling axes' root hashes, and a + /// per-ancestor attestation chain up to the grovedb root — so the + /// client reconstructs the platform root hash from it. The Merk + /// query (the encoded bounds and walk direction) and the limit are + /// echoed and re-checked by grovedb's verifier against the client's + /// own reconstruction via [`AxisRangeBounds::merk_query`] — which is + /// why the bounds are validated rather than clamped upstream, and + /// why completeness needs no extra machinery: a Merk range proof + /// over a sorted keyspace commits its boundaries, so an in-range + /// group the server omitted fails reconstruction. + /// + /// Verified by + /// [`DriveDocumentHavingQuery::verify_having_range_proof`](crate::query::DriveDocumentHavingQuery::verify_having_range_proof). + pub fn execute_range_with_proof( + &self, + drive: &Drive, + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result, Error> { + let grove_version = &platform_version.drive.grove_version; + let path = self.indexed_property_name_tree_path()?; + let path_refs: Vec<&[u8]> = path.iter().map(|segment| segment.as_slice()).collect(); + let secondary_query = self.bounds.merk_query(self.descending); + + // Same destructure-don't-unwrap rationale as the no-proof arm. + let CostContext { value, cost: _ } = match self.bounds.axis() { + RankedAxis::Count => drive.grove.prove_indexed_count_query( + path_refs.as_slice(), + secondary_query, + Some(self.limit), + transaction, + grove_version, + ), + RankedAxis::Sum => drive.grove.prove_indexed_sum_query( + path_refs.as_slice(), + secondary_query, + Some(self.limit), + transaction, + grove_version, + ), + RankedAxis::Avg => drive.grove.prove_indexed_avg_query( + path_refs.as_slice(), + secondary_query, + Some(self.limit), + transaction, + grove_version, + ), + }; + value.map_err(|e| Error::GroveDB(Box::new(e))) + } +} diff --git a/packages/rs-drive/src/query/drive_document_having_query/executors.rs b/packages/rs-drive/src/query/drive_document_having_query/executors.rs new file mode 100644 index 00000000000..97793ee923f --- /dev/null +++ b/packages/rs-drive/src/query/drive_document_having_query/executors.rs @@ -0,0 +1,117 @@ +//! Per-mode having-range executors on `impl Drive`, plus the shared +//! mode-to-query resolution. The dispatcher +//! ([`super::drive_dispatcher`]) picks between the two executors on the +//! request's `prove` flag. +//! +//! Index resolution reuses the ranked surface's covering-index picker +//! ([`find_ranked_index_for_axis`]) — both surfaces read the same +//! indexed tree, and sharing the picker is what guarantees a proof and +//! an unproven read are about the same subtree. + +use super::super::drive_document_ranked_query::index_picker::find_ranked_index_for_axis; +use super::super::drive_document_ranked_query::RankedEntry; +use super::{DocumentHavingMode, DriveDocumentHavingQuery}; +use crate::drive::Drive; +use crate::error::query::QuerySyntaxError; +use crate::error::Error; +use dpp::data_contract::document_type::accessors::DocumentTypeV0Getters; +use dpp::data_contract::document_type::{DocumentTypeRef, Index}; +use dpp::version::PlatformVersion; +use grovedb::TransactionArg; +use std::collections::BTreeMap; + +/// Resolve a validated [`DocumentHavingMode`] against a document type's +/// indexes into the executable [`DriveDocumentHavingQuery`]. +/// +/// `indexes` is threaded in separately for the same lifetime reason as +/// the ranked resolver: the returned query's `&'a Index` must outlive +/// this frame. Callers pass `document_type.indexes()`. +pub(super) fn having_query_for_mode<'a>( + contract_id: [u8; 32], + document_type: DocumentTypeRef<'a>, + document_type_name: String, + indexes: &'a BTreeMap, + mode: &DocumentHavingMode, +) -> Result, Error> { + let axis = mode.bounds.axis(); + let index = find_ranked_index_for_axis( + indexes, + &mode.group_by_property, + axis, + &mode.aggregate_field, + ) + .ok_or_else(|| { + Error::Query(QuerySyntaxError::WhereClauseOnNonIndexedProperty(format!( + "no ranked index covers `group_by = [{}]` on the {:?} axis: a `having` bound \ + is served from that axis's pre-sorted secondary, so the document type needs \ + a single-property index on `{}` declaring `{}`{}", + mode.group_by_property, + axis, + mode.group_by_property, + axis.required_index_keyword(), + if mode.aggregate_field.is_empty() { + String::new() + } else { + format!(" with `summable: \"{}\"`", mode.aggregate_field) + } + ))) + })?; + Ok(DriveDocumentHavingQuery { + document_type, + contract_id, + document_type_name, + index, + bounds: mode.bounds, + descending: mode.descending, + limit: mode.limit, + }) +} + +impl Drive { + /// One page of groups matching a having bound, read without a proof. + pub fn execute_document_having_range_no_proof( + &self, + contract_id: [u8; 32], + document_type: DocumentTypeRef, + document_type_name: String, + mode: &DocumentHavingMode, + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result, Error> { + let indexes = document_type.indexes(); + let having_query = having_query_for_mode( + contract_id, + document_type, + document_type_name, + indexes, + mode, + )?; + having_query.execute_range_no_proof(self, transaction, platform_version) + } + + /// Proof of one page of groups matching a having bound. + /// + /// The client verifies it with + /// [`DriveDocumentHavingQuery::verify_having_range_proof`](crate::query::DriveDocumentHavingQuery::verify_having_range_proof), + /// reconstructing the same query from the same contract — which is + /// why index resolution is shared with the no-proof executor. + pub fn execute_document_having_range_proof( + &self, + contract_id: [u8; 32], + document_type: DocumentTypeRef, + document_type_name: String, + mode: &DocumentHavingMode, + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result, Error> { + let indexes = document_type.indexes(); + let having_query = having_query_for_mode( + contract_id, + document_type, + document_type_name, + indexes, + mode, + )?; + having_query.execute_range_with_proof(self, transaction, platform_version) + } +} diff --git a/packages/rs-drive/src/query/drive_document_having_query/mod.rs b/packages/rs-drive/src/query/drive_document_having_query/mod.rs new file mode 100644 index 00000000000..1b554131b35 --- /dev/null +++ b/packages/rs-drive/src/query/drive_document_having_query/mod.rs @@ -0,0 +1,284 @@ +//! Types and module structure for the **boolean-`HAVING` range** document +//! query — `SELECT GROUP BY HAVING +//! [ORDER BY ASC|DESC] LIMIT n`. +//! +//! A having-range query answers "which groups' aggregate falls inside a +//! value bound?" ("hashtags with more than 100 posts") in `O(log n + k)` +//! with a proof, by range-reading the same per-axis *secondary* Merk the +//! ranked query walks (grovedb PR #657): the secondary is keyed by +//! `(sort_key ‖ group_key)` with an order-preserving sort-key encoding, +//! so an inclusive numeric bound on the aggregate is a contiguous byte +//! range in the secondary's keyspace. The same contract opt-in applies — +//! `rankedCountable` / `rankedSummable` / `rankedAverageable` (meta schema +//! v3 / PV14) — and a `HAVING` on an axis the index does not declare is +//! rejected, because serving it would mean walking every group. +//! +//! The implementation mirrors [`super::drive_document_ranked_query`] +//! sibling-for-sibling and reuses its axis / entry / pagination types +//! ([`RankedAxis`], [`RankedEntry`], [`RankedEntryValue`], +//! [`super::RankedPaginationInputs`]) and its covering-index picker — +//! both surfaces read the same tree, so sharing the resolution logic is +//! what keeps them provably about the same subtree: +//! - [`mode_detection`] — request-shape validation + the versioned +//! `(select, group_by, having, order_by, limit)` → +//! [`DocumentHavingMode`] resolution, including the operator → +//! inclusive-bounds translation. +//! - [`execute_range`] — the two executors on +//! [`DriveDocumentHavingQuery`] (no-proof read, proof generation). +//! - [`executors`] — the `impl Drive` wrappers the dispatcher calls. +//! - [`drive_dispatcher`] — [`DocumentHavingRequest`] / +//! [`DocumentHavingResponse`] and +//! [`crate::drive::Drive::execute_document_having_request`]. +//! - [`tests`] (cfg `server` + `test`) — unit + integration tests. +//! +//! ## What makes this query shape different from ranked +//! +//! Ranked addresses groups by **rank position** (`k` best, starting at +//! rank `offset`); having-range addresses them by **value bound** +//! (`aggregate ∈ [lo, hi]`). Three consequences: +//! +//! 1. **The bound is part of the proof contract.** The grovedb envelope +//! for a range read echoes the Merk query itself, and the verifier +//! re-builds that query from the request's bounds +//! ([`AxisRangeBounds::merk_query`]) — so prover and verifier must +//! share one bounds-to-query translation, exactly as they share the +//! grove path. Completeness comes from the Merk range proof: the +//! boundary commitments show no in-range group was omitted. +//! 2. **No `OFFSET`, no `start_at` — and no full pagination.** The +//! range primitives take a limit but no skip, and a request carrying +//! either knob is rejected loudly. A page cut at `limit` can only be +//! continued past **distinct** aggregate values, by tightening the +//! bound past the last value seen; a cut that lands **inside a tie** +//! (several groups sharing the boundary aggregate) cannot be +//! continued at all — moving the threshold past the tied value skips +//! the uncollected tied groups, and keeping it returns the same +//! page. Enumerating through a tie wider than [`MAX_HAVING_LIMIT`] +//! needs a cursor on the `(sort_key ‖ group_key)` composite +//! keyspace, a future capability; until then, size `limit` above the +//! widest tie the data can produce, or accept the cut. +//! 3. **Entry order is axis order in the walk direction.** Ascending by +//! default (`ORDER BY` is optional here — the bound, not the +//! ordering, is the point of the query); an explicit `ORDER BY` on +//! the selected aggregate flips the walk. Ties break by group key in +//! the direction of the walk, same as ranked. + +#[cfg(any(feature = "server", feature = "verify"))] +use dpp::data_contract::document_type::{DocumentTypeRef, Index}; + +#[cfg(any(feature = "server", feature = "verify"))] +use super::drive_document_ranked_query::{ + path::indexed_property_name_tree_path_for_index, RankedAxis, +}; +#[cfg(any(feature = "server", feature = "verify"))] +use crate::error::Error; +#[cfg(any(feature = "server", feature = "verify"))] +use grovedb::element::indexed::{encode_avg_sort_key, encode_count_sort_key, encode_sum_sort_key}; +#[cfg(any(feature = "server", feature = "verify"))] +use grovedb::Query; + +#[cfg(any(feature = "server", feature = "verify"))] +pub mod mode_detection; + +// Server-side execution paths. +#[cfg(feature = "server")] +pub mod drive_dispatcher; +#[cfg(feature = "server")] +pub mod execute_range; +#[cfg(feature = "server")] +pub mod executors; + +#[cfg(feature = "server")] +pub use drive_dispatcher::{DocumentHavingRequest, DocumentHavingResponse}; + +#[cfg(all(feature = "server", test))] +mod tests; + +/// Hard ceiling on a having-range request's `LIMIT`. Same value and same +/// rationale as [`super::drive_document_ranked_query::MAX_RANKED_LIMIT`]: +/// the proof commits one secondary entry per returned group, so proof +/// bytes grow linearly in the limit, and the ceiling is a hard rejection +/// rather than a clamp because the limit is echoed in the proof envelope +/// and re-checked by the verifier. +#[cfg(any(feature = "server", feature = "verify"))] +pub const MAX_HAVING_LIMIT: u16 = 100; + +/// Inclusive numeric bounds on one axis of an indexed tree — the resolved +/// form of a `HAVING ` clause. +/// +/// One variant per axis because the three axes have three value types +/// (`u64` count, `i64` sum, `i128` fixed-point average) and the bound +/// arithmetic (operator translation, successor/predecessor at exclusive +/// bounds) must be exact in the axis's own domain. Both bounds are +/// **inclusive**; the operator translation in +/// [`mode_detection`] normalizes every supported operator to this form, +/// rejecting translations that would overflow (`> MAX`) or invert +/// (`lo > hi`) instead of serving a silently-empty range. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +#[cfg(any(feature = "server", feature = "verify"))] +pub enum AxisRangeBounds { + /// `COUNT(*) ∈ [lo, hi]`. + Count { + /// Inclusive lower bound. + lo: u64, + /// Inclusive upper bound. + hi: u64, + }, + /// `SUM(field) ∈ [lo, hi]`. + Sum { + /// Inclusive lower bound. + lo: i64, + /// Inclusive upper bound. + hi: i64, + }, + /// `AVG(field) ∈ [lo, hi]`, in the fixed-point domain described on + /// [`super::drive_document_ranked_query::RANKED_AVG_SCALE`]. + Avg { + /// Inclusive lower bound (fixed point). + lo: i128, + /// Inclusive upper bound (fixed point). + hi: i128, + }, +} + +#[cfg(any(feature = "server", feature = "verify"))] +impl AxisRangeBounds { + /// The axis these bounds constrain. + pub fn axis(&self) -> RankedAxis { + match self { + AxisRangeBounds::Count { .. } => RankedAxis::Count, + AxisRangeBounds::Sum { .. } => RankedAxis::Sum, + AxisRangeBounds::Avg { .. } => RankedAxis::Avg, + } + } + + /// The bounds as a byte range over the axis secondary's keyspace: + /// `(inclusive_lower, exclusive_upper)`, with `None` for an upper + /// bound at the axis's type maximum (no representable successor — + /// the range is unbounded above). + /// + /// Secondary keys are `(sort_key ‖ group_key)` with order-preserving + /// fixed-width sort keys, so the inclusive numeric range `[lo, hi]` + /// is exactly the byte range `[encode(lo), encode(hi + 1))`: the + /// exclusive upper at the *next* sort key admits every group-key + /// suffix under `hi` and nothing above it. This mirrors — and must + /// stay identical to — the bound construction inside grovedb's + /// `indexed_*_range` read primitives, so the no-proof read and the + /// proved read answer the same question. + /// + /// The `+ 1` cannot overflow: the `hi == MAX` case returns `None` + /// first. + pub fn secondary_key_bounds(&self) -> (Vec, Option>) { + match *self { + AxisRangeBounds::Count { lo, hi } => ( + encode_count_sort_key(lo).to_vec(), + (hi != u64::MAX).then(|| encode_count_sort_key(hi + 1).to_vec()), + ), + AxisRangeBounds::Sum { lo, hi } => ( + encode_sum_sort_key(lo).to_vec(), + (hi != i64::MAX).then(|| encode_sum_sort_key(hi + 1).to_vec()), + ), + AxisRangeBounds::Avg { lo, hi } => ( + encode_avg_sort_key(lo).to_vec(), + (hi != i128::MAX).then(|| encode_avg_sort_key(hi + 1).to_vec()), + ), + } + } + + /// The Merk query over the axis secondary that reads exactly these + /// bounds, walking in the requested direction. + /// + /// This is the **prover/verifier-agreement artifact** of the having + /// surface: grovedb's range-proof envelope is generated against this + /// query and verified against the verifier's own reconstruction of + /// it, so both sides must build it from the same bounds through this + /// one function — a divergence surfaces as a failed verification, + /// not a wrong answer. + pub fn merk_query(&self, descending: bool) -> Query { + let (lower, upper) = self.secondary_key_bounds(); + let mut query = Query::new_with_direction(!descending); + match upper { + Some(upper) => query.insert_range(lower..upper), + None => query.insert_range_from(lower..), + } + query + } +} + +/// The resolved shape of a having-range request: the bounds (which carry +/// the axis), the walk direction, the limit, and the `(group property, +/// aggregate field)` pair the index picker needs. +/// +/// Produced by [`mode_detection::detect_having_mode`]. Parallels +/// [`super::drive_document_ranked_query::DocumentRankedMode`]. +#[derive(Debug, Clone, PartialEq, Eq)] +#[cfg(any(feature = "server", feature = "verify"))] +pub struct DocumentHavingMode { + /// Inclusive bounds on the aggregate, in the axis's own domain. + pub bounds: AxisRangeBounds, + /// Walk direction: `true` reads matching groups from the largest + /// aggregate down. Defaults to `false` (ascending) when the request + /// carries no `ORDER BY`. + pub descending: bool, + /// Maximum number of matching groups to return — + /// `1 ..= MAX_HAVING_LIMIT`, required. + pub limit: u16, + /// The single `GROUP BY` property; must be the covering ranked + /// index's only property. + pub group_by_property: String, + /// The field the aggregate applies to. Empty for `COUNT(*)`; the + /// index's `summable` property for `SUM` / `AVG`. + pub aggregate_field: String, +} + +/// A resolved having-range query. Shared by the prover and the verifier — +/// both build the grove path through +/// [`DriveDocumentHavingQuery::indexed_property_name_tree_path`] and the +/// secondary query through [`AxisRangeBounds::merk_query`], so the two +/// cannot drift on which subtree or which range the proof is about. +#[derive(Debug, Clone)] +#[cfg(any(feature = "server", feature = "verify"))] +pub struct DriveDocumentHavingQuery<'a> { + /// The document type being filtered. + pub document_type: DocumentTypeRef<'a>, + /// The contract id (32 bytes). Separate from `document_type` so the + /// verifier can build the query without the full contract. + pub contract_id: [u8; 32], + /// The document type name — a path segment. + pub document_type_name: String, + /// The covering ranked index. Single-property by construction; its + /// one property is both the `GROUP BY` property and the last path + /// segment. + pub index: &'a Index, + /// Inclusive bounds on the aggregate. Carry the axis; the index must + /// declare the matching `ranked_*` flag. + pub bounds: AxisRangeBounds, + /// `true` walks the secondary from the largest matching aggregate + /// down. Tie ordering is by group key in the direction of the walk, + /// exactly as on the ranked surface. + pub descending: bool, + /// Maximum number of matching groups to return. Fewer entries come + /// back when fewer groups fall inside the bounds; that is not an + /// error. **More matching groups than `limit` are silently cut at + /// `limit`** — the walk stops, and nothing marks the cut. A caller + /// can continue past *distinct* aggregate values by tightening the + /// bound, but a cut inside a **tie** cannot be continued (see the + /// module docs): groups tied at the boundary aggregate that fell + /// past the limit stay unreachable until a composite-key cursor + /// exists, so size the limit above the widest expected tie. + pub limit: u16, +} + +#[cfg(any(feature = "server", feature = "verify"))] +impl DriveDocumentHavingQuery<'_> { + /// Path of the terminal property-name tree the axis secondary hangs + /// off — identical to the ranked surface's path, because both read + /// the same indexed tree. See + /// [`DriveDocumentRankedQuery::indexed_property_name_tree_path`](super::drive_document_ranked_query::DriveDocumentRankedQuery::indexed_property_name_tree_path). + pub fn indexed_property_name_tree_path(&self) -> Result>, Error> { + indexed_property_name_tree_path_for_index( + &self.contract_id, + &self.document_type_name, + self.index, + ) + } +} diff --git a/packages/rs-drive/src/query/drive_document_having_query/mode_detection/mod.rs b/packages/rs-drive/src/query/drive_document_having_query/mode_detection/mod.rs new file mode 100644 index 00000000000..274dda5a872 --- /dev/null +++ b/packages/rs-drive/src/query/drive_document_having_query/mode_detection/mod.rs @@ -0,0 +1,67 @@ +//! Request-shape validation for the having-range query, and the versioned +//! `(select, group_by, having, order_by, limit)` → [`DocumentHavingMode`] +//! resolution — including the operator → inclusive-bounds translation +//! that turns a `HAVING ` clause into an +//! [`AxisRangeBounds`]. +//! +//! Pure functions on the request shape — no Drive, no contract, no +//! indexes. Available under `server` and `verify` for the same reason as +//! [`super::super::drive_document_ranked_query::mode_detection`]: both +//! sides must agree on which requests are well-formed and on the exact +//! bounds a well-formed one resolves to, because the bounds are echoed +//! (as a Merk query) inside the proof envelope. +//! +//! Versioned through +//! `platform_version.drive.methods.document.query.detect_having_mode` — +//! the accepted grammar is a consensus-adjacent contract on the query +//! surface, so relaxing it later (multi-clause `HAVING`, `IN`, a +//! pagination cursor) lands behind a method-version bump. + +use super::super::drive_document_ranked_query::RankedPaginationInputs; +use super::{AxisRangeBounds, DocumentHavingMode, MAX_HAVING_LIMIT}; +use crate::error::query::QuerySyntaxError; +use crate::error::Error; +use crate::query::having::HavingClause; +use crate::query::projection::SelectProjection; +use crate::query::{OrderClause, WhereClause}; +use dpp::version::PlatformVersion; + +/// Versioned entry point. Routes through +/// `platform_version.drive.methods.document.query.detect_having_mode`; +/// today only `0` is defined and maps to [`detect_having_mode_v0`] +/// verbatim. +#[allow(clippy::too_many_arguments)] +pub fn detect_having_mode( + select: &SelectProjection, + group_by: &[String], + having: &[HavingClause], + order_by: &[OrderClause], + where_clauses: &[WhereClause], + pagination: RankedPaginationInputs, + platform_version: &PlatformVersion, +) -> Result { + match platform_version + .drive + .methods + .document + .query + .detect_having_mode + { + 0 => detect_having_mode_v0( + select, + group_by, + having, + order_by, + where_clauses, + pagination, + ), + version => Err(Error::Query(QuerySyntaxError::Unsupported(format!( + "detect_having_mode: unknown method version {version}; only 0 is supported" + )))), + } +} + +mod v0; +// Re-exported so the dispatcher's callers (`drive_dispatcher`, the +// test suites) keep addressing the frozen grammar by its old path. +pub use v0::detect_having_mode_v0; diff --git a/packages/rs-drive/src/query/drive_document_having_query/mode_detection/v0/mod.rs b/packages/rs-drive/src/query/drive_document_having_query/mode_detection/v0/mod.rs new file mode 100644 index 00000000000..d8ff3d4f4b4 --- /dev/null +++ b/packages/rs-drive/src/query/drive_document_having_query/mode_detection/v0/mod.rs @@ -0,0 +1,674 @@ +//! Feature version 0 of the grammar — the frozen implementation +//! behind the `mode_detection` dispatcher. Everything private in +//! this file is a v0 internal: a later grammar version gets its own +//! `vN/` sibling rather than editing this one. + +use super::{AxisRangeBounds, DocumentHavingMode, MAX_HAVING_LIMIT}; +use crate::error::query::QuerySyntaxError; +use crate::error::Error; +use crate::query::drive_document_ranked_query::mode_detection::ranked_order_key; +use crate::query::drive_document_ranked_query::{RankedAxis, RankedPaginationInputs}; +use crate::query::having::{ + HavingAggregateFunction, HavingClause, HavingOperator, HavingRightOperand, +}; +use crate::query::projection::{SelectFunction, SelectProjection}; +use crate::query::{OrderClause, WhereClause}; +use dpp::platform_value::Value; +use grovedb::element::indexed::AVG_FIXED_POINT_SCALE; + +/// v0 of the having-range request grammar. +/// +/// Accepts exactly: +/// +/// ```text +/// SELECT COUNT(*) GROUP BY p HAVING COUNT(*) [ORDER BY $count [ASC|DESC]] LIMIT n +/// SELECT SUM(f) GROUP BY p HAVING SUM(f) [ORDER BY f [ASC|DESC]] LIMIT n +/// SELECT AVG(f) GROUP BY p HAVING AVG(f) [ORDER BY f [ASC|DESC]] LIMIT n +/// ``` +/// +/// with no `WHERE`, no `OFFSET`, no `START AT` / `START AFTER`, exactly +/// one `GROUP BY` property, exactly one `HAVING` clause whose aggregate +/// **is the selected aggregate** (same function, same field), an operator +/// from the contiguous-range family (`=`, `>`, `>=`, `<`, `<=`, and the +/// four `BETWEEN*` variants — `!=` and `IN` describe non-contiguous +/// ranges and are rejected as not yet supported), at most one `ORDER BY` +/// clause naming the selected aggregate, and `1 ≤ n ≤` +/// [`MAX_HAVING_LIMIT`]. +/// +/// The single-clause / same-aggregate restriction is what makes the +/// query a *range read*: one clause on the selected aggregate is one +/// contiguous slice of one axis secondary. A second clause (implicit +/// AND) or a clause on a different aggregate would need a per-candidate +/// post-check against the primary — a future capability, rejected loudly +/// today. +/// +/// Worked examples: +/// +/// ```text +/// -- hashtags with more than 100 posts, biggest first +/// SELECT COUNT(*) GROUP BY hashtag HAVING $count > 100 ORDER BY $count DESC LIMIT 100 +/// +/// -- restaurants averaging a grade of at least 4 +/// SELECT AVG(grade) GROUP BY restaurantId HAVING grade >= 4 LIMIT 50 +/// +/// -- donors whose lifetime total sits between two bounds +/// SELECT SUM(amount) GROUP BY donorId HAVING amount BETWEEN 1000 AND 5000 LIMIT 100 +/// ``` +/// +/// Everything the grammar rejects is rejected *loudly* rather than +/// normalized away — including operator translations that produce an +/// empty range (`> u64::MAX`, `BETWEEN 10 AND 5`): a bound that cannot +/// match any group is a caller error, and silently proving an empty page +/// would hide it. +pub fn detect_having_mode_v0( + select: &SelectProjection, + group_by: &[String], + having: &[HavingClause], + order_by: &[OrderClause], + where_clauses: &[WhereClause], + pagination: RankedPaginationInputs, +) -> Result { + // ---- GROUP BY: exactly one property ---------------------------- + // + // Same contract as the ranked surface: ranked indexes are + // single-property, and the sole property is what the secondary's + // group keys are. + if group_by.len() != 1 { + return Err(Error::Query(QuerySyntaxError::InvalidParameter(format!( + "having-range queries require exactly one `group_by` property (the ranked \ + index's only property); got {}. Ranked indexes are single-property — compound \ + ranked indexes are rejected at contract-parse time — so there is no compound \ + grouping to filter over.", + group_by.len() + )))); + } + let group_by_property = group_by[0].clone(); + if group_by_property.is_empty() { + return Err(Error::Query(QuerySyntaxError::InvalidParameter( + "having-range queries require a non-empty `group_by` property name".to_string(), + ))); + } + + // ---- SELECT: the axis, and the field it aggregates -------------- + // + // Same axis resolution as the ranked surface, because the same + // three secondaries serve both. + let (axis, aggregate_field) = match (select.function, select.field.as_str()) { + (SelectFunction::Count, "") => (RankedAxis::Count, String::new()), + (SelectFunction::Count, field) => { + return Err(Error::Query(QuerySyntaxError::Unsupported(format!( + "having-range queries support `COUNT(*)` but not `COUNT({field})`; the \ + count axis counts documents per group, which is what `COUNT(*)` means. \ + Drop the field to filter by group size." + )))); + } + (SelectFunction::Sum, "") | (SelectFunction::Avg, "") => { + return Err(Error::Query(QuerySyntaxError::InvalidParameter( + "`SUM` / `AVG` having-range queries require a non-empty select field naming \ + the index's `summable` property" + .to_string(), + ))); + } + (SelectFunction::Sum, field) => (RankedAxis::Sum, field.to_string()), + (SelectFunction::Avg, field) => (RankedAxis::Avg, field.to_string()), + (other, _) => { + return Err(Error::Query(QuerySyntaxError::Unsupported(format!( + "having-range queries support `COUNT(*)`, `SUM(field)` and `AVG(field)` \ + selects; got {other:?}. The bound is served from an indexed tree's \ + per-axis secondary, and grovedb maintains exactly those three axes." + )))); + } + }; + + // ---- HAVING: exactly one clause, on the selected aggregate ------ + let clause = match having { + [only] => only, + [] => { + return Err(Error::Query(QuerySyntaxError::InvalidParameter( + "having-range queries require exactly one `having` clause; got none. \ + Without a bound the request is a plain grouped aggregate — drop into \ + that surface instead." + .to_string(), + ))); + } + many => { + return Err(Error::Query(QuerySyntaxError::Unsupported(format!( + "multiple `having` clauses (implicit AND) are not yet supported: got {}. \ + One clause on the selected aggregate is one contiguous slice of one axis \ + secondary; a second clause would need a per-candidate post-check against \ + the primary. Narrow to a single clause.", + many.len() + )))); + } + }; + + let clause_axis = match clause.aggregate.function { + HavingAggregateFunction::Count => RankedAxis::Count, + HavingAggregateFunction::Sum => RankedAxis::Sum, + HavingAggregateFunction::Avg => RankedAxis::Avg, + }; + if clause_axis != axis || clause.aggregate.field != select.field { + return Err(Error::Query(QuerySyntaxError::Unsupported(format!( + "the `having` clause must bound the selected aggregate itself: the select is \ + `{:?}({})` but the clause bounds `{:?}({})`. Filtering by one aggregate while \ + projecting another would need a per-candidate post-check against the primary, \ + which is not yet supported.", + select.function, + if select.field.is_empty() { + "*" + } else { + select.field.as_str() + }, + clause.aggregate.function, + if clause.aggregate.field.is_empty() { + "*" + } else { + clause.aggregate.field.as_str() + } + )))); + } + + // ---- Operator + right operand → inclusive bounds ----------------- + let HavingRightOperand::Value(right) = &clause.right; + let bounds = match axis { + RankedAxis::Count => { + let (lo, hi) = bounds_for_operator( + clause.operator, + right, + count_operand, + u64::MIN, + u64::MAX, + |v| v.checked_add(1), + |v| v.checked_sub(1), + )?; + AxisRangeBounds::Count { lo, hi } + } + RankedAxis::Sum => { + let (lo, hi) = bounds_for_operator( + clause.operator, + right, + sum_operand, + i64::MIN, + i64::MAX, + |v| v.checked_add(1), + |v| v.checked_sub(1), + )?; + AxisRangeBounds::Sum { lo, hi } + } + RankedAxis::Avg => { + let (lo, hi) = avg_bounds_for_operator(clause.operator, right)?; + AxisRangeBounds::Avg { lo, hi } + } + }; + + // ---- ORDER BY: absent (ascending default) or the aggregate ------ + // + // Optional here, unlike ranked, because the bound — not the + // ordering — is what the query is about. When present it must name + // the selected aggregate: matching groups come off a single-key + // secondary, so there is no other order the walk could serve. + let expected_order_key = ranked_order_key(select); + let descending = match order_by { + [] => false, + [only] if only.field == expected_order_key => !only.ascending, + [only] => { + return Err(Error::Query(QuerySyntaxError::Unsupported(format!( + "`HAVING … ORDER BY {}` is not supported: matching groups are read off the \ + axis secondary, so the only ordering available is the bounded aggregate \ + itself — write `ORDER BY {expected_order_key}` or omit `order_by` for \ + ascending.", + only.field + )))); + } + many => { + return Err(Error::Query(QuerySyntaxError::InvalidParameter(format!( + "having-range queries accept at most one `order_by` clause (naming the \ + selected aggregate); got {}. The axis secondary is a single-key ordering, \ + so there is no second sort key to apply.", + many.len() + )))); + } + }; + + // ---- WHERE: must be absent -------------------------------------- + // + // Identical rationale to the ranked surface: single-property + // indexes have no equality prefix to narrow, and the secondary is + // ordered by aggregate, not by group key. + if !where_clauses.is_empty() { + return Err(Error::Query( + QuerySyntaxError::InvalidWhereClauseComponents( + "having-range queries do not accept `where` clauses: ranked indexes are \ + single-property, so there is no equality prefix to narrow, and the axis \ + secondary is ordered by aggregate rather than by group key — it cannot \ + bound a filtered subset. Bound the whole index, or add a narrower index.", + ), + )); + } + + // ---- LIMIT: required, 1 ..= MAX_HAVING_LIMIT --------------------- + // + // Required rather than defaulted for the same reason as ranked: the + // limit is echoed inside the proof envelope and re-checked by the + // verifier, so there is no server default a client could reproduce. + // Required *especially* here, because a threshold can match + // unboundedly many groups. + let limit = pagination.limit.ok_or_else(|| { + Error::Query(QuerySyntaxError::InvalidLimit(format!( + "having-range queries require an explicit `limit` (1 ..= {MAX_HAVING_LIMIT}): \ + a bound can match any number of groups, the walk stops at `limit`, and the \ + limit is echoed in the proof envelope and re-checked by the verifier, so \ + there is no server-side default a client could reproduce." + ))) + })?; + if limit == 0 { + return Err(Error::Query(QuerySyntaxError::InvalidLimit(format!( + "`LIMIT 0` selects nothing; having-range queries require 1 ≤ limit ≤ {MAX_HAVING_LIMIT}" + )))); + } + if limit > MAX_HAVING_LIMIT as u32 { + return Err(Error::Query(QuerySyntaxError::InvalidLimit(format!( + "`LIMIT {limit}` exceeds the having-range ceiling of {MAX_HAVING_LIMIT}; the \ + proof commits one secondary entry per returned group, so its size grows \ + linearly in the limit. The ceiling is a hard limit, not a clamp, because the \ + limit is echoed in the proof envelope and re-checked by the verifier. Narrow \ + the bound to shrink the result set." + )))); + } + // Bounded by MAX_HAVING_LIMIT (a u16) immediately above. + let limit = limit as u16; + + // ---- OFFSET / START AT: must be absent --------------------------- + // + // The range primitives take a limit but no skip, and unlike the + // ranked walk there is no counted-commitment shortcut for "skip m + // matching groups" — pagination of an over-long match set is a + // future cursor capability on the `(sort_key ‖ group_key)` + // keyspace, not an emulated offset. Rejected loudly, `OFFSET 0` + // included: a caller writing any offset asked for pagination + // semantics this surface does not have. + if pagination.offset.is_some() { + return Err(Error::Query(QuerySyntaxError::InvalidLimit( + "having-range queries do not accept `offset`: matching groups are read from \ + the bound's start and cut at `limit`. To reach deeper matches, tighten the \ + bound past the last aggregate value already seen — noting that a page cut \ + inside a tie (several groups sharing the boundary aggregate) cannot be \ + continued that way; size `limit` above the widest expected tie." + .to_string(), + ))); + } + if pagination.has_start_at { + return Err(Error::Query(QuerySyntaxError::InvalidLimit( + "having-range queries do not accept `start_at` / `start_after`: the cursor \ + names a document id, which does not appear in a keyspace sorted by \ + aggregate." + .to_string(), + ))); + } + + Ok(DocumentHavingMode { + bounds, + descending, + limit, + group_by_property, + aggregate_field, + }) +} + +/// Translate `(operator, right operand)` into inclusive `[lo, hi]` +/// bounds in one axis's value domain. +/// +/// `operand` extracts a single scalar from a [`Value`] in that domain; +/// `succ` / `pred` are the domain's checked successor / predecessor, +/// used to normalize the exclusive operators (`>`, `<`, the `BETWEEN` +/// exclusions) onto inclusive bounds. A `succ`/`pred` that overflows +/// means the operator excludes the entire domain past its own extreme +/// (`> MAX`, `< MIN`) — rejected, like every other empty translation, +/// rather than served as a proof of nothing. +fn bounds_for_operator( + operator: HavingOperator, + right: &Value, + operand: impl Fn(&Value) -> Result, + min: T, + max: T, + succ: impl Fn(T) -> Option, + pred: impl Fn(T) -> Option, +) -> Result<(T, T), Error> { + let scalar = || operand(right); + let pair = || -> Result<(T, T), Error> { + let Some(items) = right.as_array() else { + return Err(Error::Query(QuerySyntaxError::InvalidParameter(format!( + "`{operator:?}` requires a 2-element list operand `[lower, upper]`; got a \ + non-list value" + )))); + }; + let [lower, upper] = items.as_slice() else { + return Err(Error::Query(QuerySyntaxError::InvalidParameter(format!( + "`{operator:?}` requires a 2-element list operand `[lower, upper]`; got {} \ + element(s)", + items.len() + )))); + }; + Ok((operand(lower)?, operand(upper)?)) + }; + let strictly_above = |v: T| { + succ(v).ok_or_else(|| { + Error::Query(QuerySyntaxError::InvalidParameter(format!( + "`{operator:?} {v}` matches no possible aggregate value: {v} is the \ + largest value the aggregate can take" + ))) + }) + }; + let strictly_below = |v: T| { + pred(v).ok_or_else(|| { + Error::Query(QuerySyntaxError::InvalidParameter(format!( + "`{operator:?} {v}` matches no possible aggregate value: {v} is the \ + smallest value the aggregate can take" + ))) + }) + }; + + let (lo, hi) = match operator { + HavingOperator::Equal => { + let v = scalar()?; + (v, v) + } + HavingOperator::GreaterThan => (strictly_above(scalar()?)?, max), + HavingOperator::GreaterThanOrEquals => (scalar()?, max), + HavingOperator::LessThan => (min, strictly_below(scalar()?)?), + HavingOperator::LessThanOrEquals => (min, scalar()?), + HavingOperator::Between => pair()?, + HavingOperator::BetweenExcludeBounds => { + let (lower, upper) = pair()?; + (strictly_above(lower)?, strictly_below(upper)?) + } + HavingOperator::BetweenExcludeLeft => { + let (lower, upper) = pair()?; + (strictly_above(lower)?, upper) + } + HavingOperator::BetweenExcludeRight => { + let (lower, upper) = pair()?; + (lower, strictly_below(upper)?) + } + HavingOperator::NotEqual | HavingOperator::In => { + return Err(Error::Query(QuerySyntaxError::Unsupported(format!( + "`{operator:?}` is not yet supported in having-range queries: it describes \ + a non-contiguous set of aggregate values, and the axis secondary serves \ + one contiguous range per request. Use a range operator, or issue one \ + request per contiguous range." + )))); + } + }; + + if lo > hi { + return Err(Error::Query(QuerySyntaxError::InvalidParameter(format!( + "the `having` bound resolves to the empty range [{lo}, {hi}] (lower above \ + upper), which matches no group; fix the operand" + )))); + } + Ok((lo, hi)) +} + +/// Extract a count operand: a non-negative integer. +fn count_operand(value: &Value) -> Result { + value.to_integer::().map_err(|_| { + Error::Query(QuerySyntaxError::InvalidParameter(format!( + "a `COUNT(*)` having bound must be a non-negative integer; got {value}" + ))) + }) +} + +/// Extract a sum operand: a signed integer in `i64` range. +fn sum_operand(value: &Value) -> Result { + value.to_integer::().map_err(|_| { + Error::Query(QuerySyntaxError::InvalidParameter(format!( + "a `SUM(field)` having bound must be an integer within i64 range; got {value}" + ))) + }) +} + +/// An `AVG` operand scaled into the axis's fixed-point domain, kept as +/// `(⌊t × SCALE⌋, is the product exactly that integer?)` so the operator +/// translation can pick the correct floor or ceiling per bound. +/// +/// A plain truncated `i128` would be wrong for floats: truncation is +/// toward zero, but an inclusive lower bound needs the *ceiling* and an +/// upper bound the *floor* — and around zero the two diverge in +/// opposite directions (`AVG >= 0.5-tick` must start at tick 1, while +/// truncation says 0; `AVG > -0.5-tick` must start at tick 0, while +/// truncate-then-increment says 1). +#[derive(Debug, Clone, Copy)] +struct ScaledAvgOperand { + /// `⌊t × SCALE⌋` — the floor (toward −∞) of the exact real product. + floor: i128, + /// Whether `t × SCALE` is exactly `floor` (the operand lands on a + /// fixed-point tick). Always true for integer operands. + exact: bool, +} + +/// Extract an average operand and scale it into the axis's fixed-point +/// domain (see +/// [`super::super::drive_document_ranked_query::RANKED_AVG_SCALE`]) +/// **exactly**. +/// +/// Integer operands scale exactly (`v × SCALE` — the product of any i64 +/// with the scale fits in `i128` by the compile-time bound next to the +/// scale constant). Float operands are decomposed into their IEEE-754 +/// `±mantissa × 2^exponent` form and the product `±mantissa × SCALE × +/// 2^exponent` is floored with integer arithmetic — never through an +/// `f64` multiplication, which loses sub-tick precision long before +/// this scale (`SCALE = 10^19 > 2^53`): the nearest-f64 rounding of +/// `t × SCALE` can land on the wrong side of a tick and silently move +/// an inclusive bound by one. +fn scaled_avg_operand(value: &Value) -> Result { + if let Some(int) = value.as_integer::() { + let floor = (int as i128) + .checked_mul(AVG_FIXED_POINT_SCALE) + .ok_or_else(|| { + Error::Query(QuerySyntaxError::InvalidParameter(format!( + "the `AVG(field)` having bound {int} does not fit the fixed-point domain" + ))) + })?; + return Ok(ScaledAvgOperand { floor, exact: true }); + } + let float = value.to_float().map_err(|_| { + Error::Query(QuerySyntaxError::InvalidParameter(format!( + "an `AVG(field)` having bound must be an integer or a float; got {value}" + ))) + })?; + if !float.is_finite() { + return Err(Error::Query(QuerySyntaxError::InvalidParameter(format!( + "an `AVG(field)` having bound must be finite; got {float}" + )))); + } + + // IEEE-754 double decomposition: float = ±mantissa × 2^exponent, + // with the implicit leading bit restored for normal numbers and the + // subnormal exponent pinned at 2^-1074. + let bits = float.to_bits(); + let negative = bits >> 63 == 1; + let raw_exponent = ((bits >> 52) & 0x7ff) as i64; + let fraction = bits & 0x000f_ffff_ffff_ffff; + let (mantissa, exponent) = if raw_exponent == 0 { + (fraction, -1074i64) + } else { + (fraction | 0x0010_0000_0000_0000, raw_exponent - 1075) + }; + if mantissa == 0 { + // ±0.0 — exactly tick zero. + return Ok(ScaledAvgOperand { + floor: 0, + exact: true, + }); + } + let out_of_domain = || { + Error::Query(QuerySyntaxError::InvalidParameter(format!( + "the `AVG(field)` having bound {float} does not fit the fixed-point domain" + ))) + }; + + // mantissa < 2^54 and SCALE < 2^64, so the product stays well under + // i128::MAX (< 2^118); only the 2^exponent factor can overflow. + let magnitude = (mantissa as i128) * AVG_FIXED_POINT_SCALE; + let signed = if negative { -magnitude } else { magnitude }; + if exponent >= 0 { + // × 2^exponent, exactly. |signed| ≥ SCALE ≥ 1, so a factor the + // domain cannot hold means the bound itself is out of domain. + if exponent >= 127 { + return Err(out_of_domain()); + } + let floor = signed + .checked_mul(1i128 << exponent) + .ok_or_else(out_of_domain)?; + Ok(ScaledAvgOperand { floor, exact: true }) + } else { + // ÷ 2^-exponent with euclidean (toward −∞) division — exactly + // the floor, with the remainder deciding exactness. + let shift = -exponent as u32; + if shift >= 127 { + // |signed| < 2^118 < 2^shift ⇒ 0 < |t × SCALE| < 1: the + // product floors to 0 (positive) or −1 (negative), and is + // never exact (mantissa is non-zero). + return Ok(ScaledAvgOperand { + floor: if negative { -1 } else { 0 }, + exact: false, + }); + } + let divisor = 1i128 << shift; + Ok(ScaledAvgOperand { + floor: signed.div_euclid(divisor), + exact: signed.rem_euclid(divisor) == 0, + }) + } +} + +/// Translate `(operator, right operand)` into inclusive `[lo, hi]` +/// bounds in the Avg axis's fixed-point domain. +/// +/// The Avg counterpart of [`bounds_for_operator`], separate because Avg +/// operands may be floats that do not land on a fixed-point tick, and +/// the correct translation is then **operator-aware**: an inclusive +/// lower bound takes the ceiling of the exact product `t × SCALE`, an +/// upper bound its floor, and the exclusive translations collapse onto +/// the inclusive ones whenever `t` sits strictly between two ticks +/// (`v > t` and `v ≥ t` admit exactly the same integers there). All of +/// it works off [`scaled_avg_operand`]'s exact `(floor, exact)` pair: +/// +/// | operator | lower bound | upper bound | +/// |---------------|------------------------|------------------------| +/// | `= t` | `t` exact on a tick — otherwise rejected: nothing can match | +/// | `> t` | `⌊t⌋ + 1` | domain max | +/// | `>= t` | `⌈t⌉` | domain max | +/// | `< t` | domain min | `⌈t⌉ − 1` | +/// | `<= t` | domain min | `⌊t⌋` | +/// | `BETWEEN*` | per-end combination of the four rows above | +/// +/// Empty translations (`> MAX`, a between pair that inverts, an +/// equality between ticks) are rejected loudly, matching +/// [`bounds_for_operator`]'s contract: a bound that cannot match any +/// group is a caller error, and silently proving an empty page would +/// hide it. +fn avg_bounds_for_operator(operator: HavingOperator, right: &Value) -> Result<(i128, i128), Error> { + let scalar = || scaled_avg_operand(right); + let pair = || -> Result<(ScaledAvgOperand, ScaledAvgOperand), Error> { + let Some(items) = right.as_array() else { + return Err(Error::Query(QuerySyntaxError::InvalidParameter(format!( + "`{operator:?}` requires a 2-element list operand `[lower, upper]`; got a \ + non-list value" + )))); + }; + let [lower, upper] = items.as_slice() else { + return Err(Error::Query(QuerySyntaxError::InvalidParameter(format!( + "`{operator:?}` requires a 2-element list operand `[lower, upper]`; got {} \ + element(s)", + items.len() + )))); + }; + Ok((scaled_avg_operand(lower)?, scaled_avg_operand(upper)?)) + }; + let past_max = || { + Error::Query(QuerySyntaxError::InvalidParameter(format!( + "the `{operator:?}` bound matches no possible aggregate value: it lies at or \ + above the largest value the aggregate can take" + ))) + }; + let past_min = || { + Error::Query(QuerySyntaxError::InvalidParameter(format!( + "the `{operator:?}` bound matches no possible aggregate value: it lies at or \ + below the smallest value the aggregate can take" + ))) + }; + + // v > t ⇔ v ≥ ⌊t⌋ + 1 whether or not t is a tick (for a tick, + // strictly above it; between ticks, the ceiling). + let exclusive_lower = |bound: ScaledAvgOperand| bound.floor.checked_add(1).ok_or_else(past_max); + // v ≥ t ⇔ v ≥ ⌈t⌉. + let inclusive_lower = |bound: ScaledAvgOperand| { + if bound.exact { + Ok(bound.floor) + } else { + bound.floor.checked_add(1).ok_or_else(past_max) + } + }; + // v < t ⇔ v ≤ ⌈t⌉ − 1 (t on a tick: strictly below it; between + // ticks: the floor). + let exclusive_upper = |bound: ScaledAvgOperand| { + if bound.exact { + bound.floor.checked_sub(1).ok_or_else(past_min) + } else { + Ok(bound.floor) + } + }; + // v ≤ t ⇔ v ≤ ⌊t⌋, which never overflows. + let inclusive_upper = |bound: ScaledAvgOperand| bound.floor; + + let (lo, hi) = match operator { + HavingOperator::Equal => { + let bound = scalar()?; + if !bound.exact { + return Err(Error::Query(QuerySyntaxError::InvalidParameter(format!( + "the `AVG(field)` equality bound {right} does not land on a fixed-point \ + tick, so no group's average can equal it; use a range operator (e.g. \ + `BETWEEN`) around the intended value, or an operand that scales exactly" + )))); + } + (bound.floor, bound.floor) + } + HavingOperator::GreaterThan => (exclusive_lower(scalar()?)?, i128::MAX), + HavingOperator::GreaterThanOrEquals => (inclusive_lower(scalar()?)?, i128::MAX), + HavingOperator::LessThan => (i128::MIN, exclusive_upper(scalar()?)?), + HavingOperator::LessThanOrEquals => (i128::MIN, inclusive_upper(scalar()?)), + HavingOperator::Between => { + let (lower, upper) = pair()?; + (inclusive_lower(lower)?, inclusive_upper(upper)) + } + HavingOperator::BetweenExcludeBounds => { + let (lower, upper) = pair()?; + (exclusive_lower(lower)?, exclusive_upper(upper)?) + } + HavingOperator::BetweenExcludeLeft => { + let (lower, upper) = pair()?; + (exclusive_lower(lower)?, inclusive_upper(upper)) + } + HavingOperator::BetweenExcludeRight => { + let (lower, upper) = pair()?; + (inclusive_lower(lower)?, exclusive_upper(upper)?) + } + HavingOperator::NotEqual | HavingOperator::In => { + return Err(Error::Query(QuerySyntaxError::Unsupported(format!( + "`{operator:?}` is not yet supported in having-range queries: it describes \ + a non-contiguous set of aggregate values, and the axis secondary serves \ + one contiguous range per request. Use a range operator, or issue one \ + request per contiguous range." + )))); + } + }; + + if lo > hi { + return Err(Error::Query(QuerySyntaxError::InvalidParameter(format!( + "the `having` bound resolves to the empty range [{lo}, {hi}] (lower above \ + upper), which matches no group; fix the operand" + )))); + } + Ok((lo, hi)) +} diff --git a/packages/rs-drive/src/query/drive_document_having_query/tests.rs b/packages/rs-drive/src/query/drive_document_having_query/tests.rs new file mode 100644 index 00000000000..924fcafc275 --- /dev/null +++ b/packages/rs-drive/src/query/drive_document_having_query/tests.rs @@ -0,0 +1,1539 @@ +//! Unit + integration tests for the having-range query surface. +//! +//! Mirrors the structure of +//! [`super::super::drive_document_ranked_query::tests`]: grammar and +//! bounds tests are pure (no Drive), execution tests run against a real +//! Drive with the shared `restaurants` fixture (see that module's docs +//! for the doctype → axis table) and documents inserted through the +//! real write path, with every proof round-tripped through +//! [`DriveDocumentHavingQuery::verify_having_range_proof`] and checked +//! against the live grovedb root hash. + +use super::mode_detection::detect_having_mode_v0; +use super::{AxisRangeBounds, MAX_HAVING_LIMIT}; +use crate::query::drive_document_ranked_query::RankedPaginationInputs; +use crate::query::having::{ + HavingAggregate, HavingAggregateFunction, HavingClause, HavingOperator, HavingRightOperand, +}; +use crate::query::projection::SelectProjection; +use crate::query::OrderClause; +use dpp::platform_value::Value; + +fn clause( + function: HavingAggregateFunction, + field: &str, + operator: HavingOperator, + right: Value, +) -> HavingClause { + HavingClause { + aggregate: HavingAggregate { + function, + field: field.to_string(), + }, + operator, + right: HavingRightOperand::Value(right), + } +} + +fn pagination(limit: u32) -> RankedPaginationInputs { + RankedPaginationInputs { + limit: Some(limit), + offset: None, + has_start_at: false, + } +} + +mod grammar { + use super::*; + + #[test] + fn count_greater_than_resolves_to_exclusive_lower_bound() { + let mode = detect_having_mode_v0( + &SelectProjection::count_star(), + &["hashtag".to_string()], + &[clause( + HavingAggregateFunction::Count, + "", + HavingOperator::GreaterThan, + Value::U64(100), + )], + &[], + &[], + pagination(10), + ) + .expect("should resolve"); + assert_eq!( + mode.bounds, + AxisRangeBounds::Count { + lo: 101, + hi: u64::MAX + } + ); + assert!(!mode.descending); + assert_eq!(mode.limit, 10); + assert_eq!(mode.group_by_property, "hashtag"); + assert_eq!(mode.aggregate_field, ""); + } + + #[test] + fn sum_between_is_inclusive_on_both_ends() { + let mode = detect_having_mode_v0( + &SelectProjection::sum("amount"), + &["donorId".to_string()], + &[clause( + HavingAggregateFunction::Sum, + "amount", + HavingOperator::Between, + Value::Array(vec![Value::I64(1000), Value::I64(5000)]), + )], + &[], + &[], + pagination(100), + ) + .expect("should resolve"); + assert_eq!(mode.bounds, AxisRangeBounds::Sum { lo: 1000, hi: 5000 }); + } + + #[test] + fn between_exclude_bounds_moves_both_ends_inward() { + let mode = detect_having_mode_v0( + &SelectProjection::count_star(), + &["hashtag".to_string()], + &[clause( + HavingAggregateFunction::Count, + "", + HavingOperator::BetweenExcludeBounds, + Value::Array(vec![Value::U64(5), Value::U64(10)]), + )], + &[], + &[], + pagination(10), + ) + .expect("should resolve"); + assert_eq!(mode.bounds, AxisRangeBounds::Count { lo: 6, hi: 9 }); + } + + #[test] + fn avg_integer_threshold_scales_exactly_into_fixed_point() { + use crate::query::drive_document_ranked_query::RANKED_AVG_SCALE; + let mode = detect_having_mode_v0( + &SelectProjection::avg("grade"), + &["restaurantId".to_string()], + &[clause( + HavingAggregateFunction::Avg, + "grade", + HavingOperator::GreaterThanOrEquals, + Value::U64(4), + )], + &[], + &[], + pagination(50), + ) + .expect("should resolve"); + assert_eq!( + mode.bounds, + AxisRangeBounds::Avg { + lo: 4 * RANKED_AVG_SCALE, + hi: i128::MAX + } + ); + } + + /// Resolve one `AVG(grade)` clause and return the bounds. + fn avg_bounds( + operator: HavingOperator, + right: Value, + ) -> Result { + detect_having_mode_v0( + &SelectProjection::avg("grade"), + &["restaurantId".to_string()], + &[clause( + HavingAggregateFunction::Avg, + "grade", + operator, + right, + )], + &[], + &[], + pagination(50), + ) + .map(|mode| mode.bounds) + } + + /// Float thresholds translate through the **exact** IEEE-754 value + /// with operator-aware floor/ceiling — never through truncation. + /// `80.5` is exactly representable (`161 × 2⁻¹`), so its scaled + /// product lands on a tick and the inclusive/exclusive translations + /// differ by exactly one, on both ends. + #[test] + fn avg_float_threshold_on_a_tick_translates_like_an_integer() { + use crate::query::drive_document_ranked_query::RANKED_AVG_SCALE; + let tick = 161 * RANKED_AVG_SCALE / 2; // 80.5 × SCALE, exact + let max = i128::MAX; + let min = i128::MIN; + for (operator, expected_lo, expected_hi) in [ + (HavingOperator::GreaterThanOrEquals, tick, max), + (HavingOperator::GreaterThan, tick + 1, max), + (HavingOperator::LessThanOrEquals, min, tick), + (HavingOperator::LessThan, min, tick - 1), + (HavingOperator::Equal, tick, tick), + ] { + assert_eq!( + avg_bounds(operator, Value::Float(80.5)).expect("80.5 scales exactly"), + AxisRangeBounds::Avg { + lo: expected_lo, + hi: expected_hi + }, + "wrong translation for {operator:?} 80.5" + ); + } + } + + /// A float threshold that falls **between** two ticks: the + /// inclusive and exclusive translations collapse onto the same + /// integer bound — the ceiling for lower bounds, the floor for + /// upper bounds. `5e-20` scales to ≈0.5 of a tick, the exact case + /// truncation used to get wrong (`AVG >= 0.5-tick` must start at + /// tick 1, not 0), and its negation exercises the + /// negative-threshold direction (`AVG > -0.5-tick` must start at + /// tick 0, not 1 — truncate-then-increment lands on 1). + #[test] + fn avg_float_threshold_between_ticks_takes_operator_aware_bounds() { + let half_tick = Value::Float(5e-20); // ≈ 0.5 of a fixed-point tick + let neg_half_tick = Value::Float(-5e-20); + let max = i128::MAX; + let min = i128::MIN; + for (operator, right, expected_lo, expected_hi) in [ + ( + HavingOperator::GreaterThanOrEquals, + half_tick.clone(), + 1, + max, + ), + (HavingOperator::GreaterThan, half_tick.clone(), 1, max), + (HavingOperator::LessThanOrEquals, half_tick.clone(), min, 0), + (HavingOperator::LessThan, half_tick.clone(), min, 0), + (HavingOperator::GreaterThan, neg_half_tick.clone(), 0, max), + ( + HavingOperator::GreaterThanOrEquals, + neg_half_tick.clone(), + 0, + max, + ), + (HavingOperator::LessThan, neg_half_tick.clone(), min, -1), + ( + HavingOperator::LessThanOrEquals, + neg_half_tick.clone(), + min, + -1, + ), + ] { + assert_eq!( + avg_bounds(operator, right.clone()).expect("between-tick thresholds resolve"), + AxisRangeBounds::Avg { + lo: expected_lo, + hi: expected_hi + }, + "wrong translation for {operator:?} {right:?}" + ); + } + + // An equality on a value between ticks can never match a + // group's average; it is rejected loudly rather than silently + // converted into a point lookup on the truncated tick. + let error = avg_bounds(HavingOperator::Equal, half_tick) + .expect_err("equality between ticks matches nothing"); + assert!( + format!("{error}").contains("does not land on a fixed-point tick"), + "the rejection must explain the tick mismatch, got: {error}" + ); + } + + #[test] + fn order_by_the_selected_aggregate_sets_direction() { + let mode = detect_having_mode_v0( + &SelectProjection::count_star(), + &["hashtag".to_string()], + &[clause( + HavingAggregateFunction::Count, + "", + HavingOperator::GreaterThan, + Value::U64(100), + )], + &[OrderClause { + field: "$count".to_string(), + ascending: false, + }], + &[], + pagination(10), + ) + .expect("should resolve"); + assert!(mode.descending); + } + + #[test] + fn ordering_by_anything_else_is_rejected() { + let result = detect_having_mode_v0( + &SelectProjection::count_star(), + &["hashtag".to_string()], + &[clause( + HavingAggregateFunction::Count, + "", + HavingOperator::GreaterThan, + Value::U64(100), + )], + &[OrderClause { + field: "hashtag".to_string(), + ascending: true, + }], + &[], + pagination(10), + ); + assert!(result.is_err(), "ordering by a schema property must fail"); + } + + #[test] + fn clause_on_a_different_aggregate_than_the_select_is_rejected() { + let result = detect_having_mode_v0( + &SelectProjection::count_star(), + &["hashtag".to_string()], + &[clause( + HavingAggregateFunction::Sum, + "amount", + HavingOperator::GreaterThan, + Value::I64(100), + )], + &[], + &[], + pagination(10), + ); + assert!(result.is_err(), "cross-aggregate having must fail"); + } + + /// `GROUP BY identityId, class HAVING AVG(grade) > 80` — compound + /// grouping — is rejected: ranked axes live on single-property + /// indexes (a contract declaring a ranked flag on a compound index + /// is already rejected at contract-parse time), so there is no + /// compound grouping for a bound to filter over. + #[test] + fn compound_group_by_is_rejected() { + let result = detect_having_mode_v0( + &SelectProjection::avg("grade"), + &["identityId".to_string(), "class".to_string()], + &[clause( + HavingAggregateFunction::Avg, + "grade", + HavingOperator::GreaterThan, + Value::U64(80), + )], + &[], + &[], + pagination(10), + ); + let error = result.expect_err("compound group_by must fail"); + assert!( + format!("{error}").contains("exactly one `group_by` property"), + "the rejection must say the surface is single-property, got: {error}" + ); + } + + #[test] + fn multiple_clauses_are_rejected() { + let single = clause( + HavingAggregateFunction::Count, + "", + HavingOperator::GreaterThan, + Value::U64(100), + ); + let result = detect_having_mode_v0( + &SelectProjection::count_star(), + &["hashtag".to_string()], + &[single.clone(), single], + &[], + &[], + pagination(10), + ); + assert!(result.is_err(), "multi-clause having must fail"); + } + + #[test] + fn not_equal_and_in_are_rejected_as_non_contiguous() { + for operator in [HavingOperator::NotEqual, HavingOperator::In] { + let result = detect_having_mode_v0( + &SelectProjection::count_star(), + &["hashtag".to_string()], + &[clause( + HavingAggregateFunction::Count, + "", + operator, + Value::U64(100), + )], + &[], + &[], + pagination(10), + ); + assert!(result.is_err(), "{operator:?} must fail"); + } + } + + #[test] + fn greater_than_the_type_maximum_is_rejected_not_served_empty() { + let result = detect_having_mode_v0( + &SelectProjection::count_star(), + &["hashtag".to_string()], + &[clause( + HavingAggregateFunction::Count, + "", + HavingOperator::GreaterThan, + Value::U64(u64::MAX), + )], + &[], + &[], + pagination(10), + ); + assert!(result.is_err(), "> u64::MAX must fail loudly"); + } + + #[test] + fn inverted_between_is_rejected() { + let result = detect_having_mode_v0( + &SelectProjection::count_star(), + &["hashtag".to_string()], + &[clause( + HavingAggregateFunction::Count, + "", + HavingOperator::Between, + Value::Array(vec![Value::U64(10), Value::U64(5)]), + )], + &[], + &[], + pagination(10), + ); + assert!(result.is_err(), "inverted bounds must fail loudly"); + } + + #[test] + fn negative_count_bound_is_rejected() { + let result = detect_having_mode_v0( + &SelectProjection::count_star(), + &["hashtag".to_string()], + &[clause( + HavingAggregateFunction::Count, + "", + HavingOperator::GreaterThan, + Value::I64(-1), + )], + &[], + &[], + pagination(10), + ); + assert!(result.is_err(), "a negative COUNT bound must fail"); + } + + #[test] + fn limit_is_required_and_capped() { + let having = [clause( + HavingAggregateFunction::Count, + "", + HavingOperator::GreaterThan, + Value::U64(100), + )]; + let select = SelectProjection::count_star(); + let group_by = ["hashtag".to_string()]; + + let missing = detect_having_mode_v0( + &select, + &group_by, + &having, + &[], + &[], + RankedPaginationInputs::default(), + ); + assert!(missing.is_err(), "a missing limit must fail"); + + let over = detect_having_mode_v0( + &select, + &group_by, + &having, + &[], + &[], + pagination(MAX_HAVING_LIMIT as u32 + 1), + ); + assert!(over.is_err(), "an over-ceiling limit must fail"); + } + + #[test] + fn offset_and_start_at_are_rejected() { + let having = [clause( + HavingAggregateFunction::Count, + "", + HavingOperator::GreaterThan, + Value::U64(100), + )]; + let select = SelectProjection::count_star(); + let group_by = ["hashtag".to_string()]; + + let with_offset = detect_having_mode_v0( + &select, + &group_by, + &having, + &[], + &[], + RankedPaginationInputs { + limit: Some(10), + offset: Some(0), + has_start_at: false, + }, + ); + assert!(with_offset.is_err(), "any offset (even 0) must fail"); + + let with_start = detect_having_mode_v0( + &select, + &group_by, + &having, + &[], + &[], + RankedPaginationInputs { + limit: Some(10), + offset: None, + has_start_at: true, + }, + ); + assert!(with_start.is_err(), "start_at must fail"); + } +} + +mod bounds { + use super::*; + use grovedb::element::indexed::{ + encode_avg_sort_key, encode_count_sort_key, encode_sum_sort_key, + }; + + #[test] + fn count_byte_bounds_bracket_the_inclusive_range() { + let bounds = AxisRangeBounds::Count { lo: 101, hi: 200 }; + let (lower, upper) = bounds.secondary_key_bounds(); + assert_eq!(lower, encode_count_sort_key(101).to_vec()); + assert_eq!(upper, Some(encode_count_sort_key(201).to_vec())); + } + + #[test] + fn unbounded_above_uses_range_from() { + let bounds = AxisRangeBounds::Count { + lo: 101, + hi: u64::MAX, + }; + let (_, upper) = bounds.secondary_key_bounds(); + assert_eq!(upper, None, "hi == MAX has no representable successor"); + + let sum_bounds = AxisRangeBounds::Sum { + lo: 0, + hi: i64::MAX, + }; + assert_eq!(sum_bounds.secondary_key_bounds().1, None); + + let avg_bounds = AxisRangeBounds::Avg { + lo: 0, + hi: i128::MAX, + }; + assert_eq!(avg_bounds.secondary_key_bounds().1, None); + } + + #[test] + fn sum_and_avg_bounds_use_the_sign_flipped_encodings() { + let sum_bounds = AxisRangeBounds::Sum { lo: -5, hi: 5 }; + let (lower, upper) = sum_bounds.secondary_key_bounds(); + assert_eq!(lower, encode_sum_sort_key(-5).to_vec()); + assert_eq!(upper, Some(encode_sum_sort_key(6).to_vec())); + + let avg_bounds = AxisRangeBounds::Avg { lo: -5, hi: 5 }; + let (lower, upper) = avg_bounds.secondary_key_bounds(); + assert_eq!(lower, encode_avg_sort_key(-5).to_vec()); + assert_eq!(upper, Some(encode_avg_sort_key(6).to_vec())); + } + + #[test] + fn merk_query_direction_follows_descending() { + let bounds = AxisRangeBounds::Count { lo: 101, hi: 200 }; + assert!(bounds.merk_query(false).left_to_right); + assert!(!bounds.merk_query(true).left_to_right); + } +} + +mod execution { + //! End-to-end behaviour against the `restaurants` fixture: the + //! dispatcher run through its public entry point (the same call + //! drive-abci makes), no-proof and proved, on all three axes. + + use super::super::drive_dispatcher::{DocumentHavingRequest, DocumentHavingResponse}; + use super::super::mode_detection::detect_having_mode; + use super::super::{AxisRangeBounds, DriveDocumentHavingQuery}; + use super::clause; + use crate::drive::Drive; + use crate::error::Error; + use crate::query::drive_document_ranked_query::index_picker::find_ranked_index_for_axis; + use crate::query::drive_document_ranked_query::{ + RankedEntry, RankedEntryValue, RankedPaginationInputs, RANKED_COUNT_ORDER_KEY, + }; + use crate::query::having::{HavingAggregateFunction, HavingClause, HavingOperator}; + use crate::query::projection::SelectProjection; + use crate::query::OrderClause; + use crate::util::object_size_info::DocumentInfo::DocumentRefInfo; + use crate::util::object_size_info::{DocumentAndContractInfo, OwnedDocumentInfo}; + use crate::util::storage_flags::StorageFlags; + use crate::util::test_helpers::setup::setup_drive_with_initial_state_structure; + use dpp::block::block_info::BlockInfo; + use dpp::data_contract::accessors::v0::DataContractV0Getters; + use dpp::data_contract::document_type::accessors::DocumentTypeV0Getters; + use dpp::data_contract::document_type::random_document::CreateRandomDocument; + use dpp::document::{Document, DocumentV0Setters}; + use dpp::platform_value::Value; + use dpp::prelude::DataContract; + use dpp::tests::json_document::json_document_to_contract; + use dpp::version::PlatformVersion; + use grovedb::element::indexed::compute_avg_fixed_point; + use std::collections::BTreeMap; + + const GROUP_PROPERTY: &str = "restaurantId"; + + fn platform_version() -> &'static PlatformVersion { + PlatformVersion::latest() + } + + fn setup_restaurants() -> (Drive, DataContract) { + let drive = setup_drive_with_initial_state_structure(None); + let pv = platform_version(); + let contract = json_document_to_contract( + "tests/supporting_files/contract/restaurants/restaurants-contract.json", + false, + pv, + ) + .expect("expected to parse the restaurants contract"); + drive + .apply_contract( + &contract, + BlockInfo::default(), + true, + StorageFlags::optional_default_as_cow(), + None, + pv, + ) + .expect("expected to apply the restaurants contract"); + (drive, contract) + } + + /// Same real-write-path insertion as the ranked suite; see its docs + /// for the disjoint-seed requirement. + fn insert_docs( + drive: &Drive, + contract: &DataContract, + document_type_name: &str, + aggregated_property: &str, + first_seed: u64, + rows: &[(&str, i64)], + ) { + let pv = platform_version(); + let document_type = contract + .document_type_for_name(document_type_name) + .unwrap_or_else(|_| panic!("{document_type_name} doctype exists")); + for (i, (restaurant, value)) in rows.iter().enumerate() { + let mut doc: Document = document_type + .random_document(Some(first_seed + i as u64), pv) + .expect("random document"); + let mut props = BTreeMap::new(); + props.insert( + GROUP_PROPERTY.to_string(), + Value::Text(restaurant.to_string()), + ); + props.insert(aggregated_property.to_string(), Value::I64(*value)); + doc.set_properties(props); + drive + .add_document_for_contract( + DocumentAndContractInfo { + owned_document_info: OwnedDocumentInfo { + document_info: DocumentRefInfo((&doc, None)), + owner_id: None, + }, + contract, + document_type, + }, + false, + BlockInfo::default(), + true, + None, + pv, + None, + ) + .unwrap_or_else(|e| { + panic!("expected to insert a {document_type_name} document: {e}") + }); + } + } + + /// One having request, minus the `prove` flag. + #[derive(Clone)] + struct HavingCase { + document_type_name: &'static str, + select: SelectProjection, + having: HavingClause, + /// `None` means no `ORDER BY` (ascending default); + /// `Some(ascending)` orders by the selected aggregate. + order_ascending: Option, + limit: Option, + } + + impl HavingCase { + fn count(operator: HavingOperator, right: Value, limit: u32) -> Self { + Self { + document_type_name: "visit", + select: SelectProjection::count_star(), + having: clause(HavingAggregateFunction::Count, "", operator, right), + order_ascending: None, + limit: Some(limit), + } + } + + fn sum(operator: HavingOperator, right: Value, limit: u32) -> Self { + Self { + document_type_name: "tip", + select: SelectProjection::sum("amount"), + having: clause(HavingAggregateFunction::Sum, "amount", operator, right), + order_ascending: None, + limit: Some(limit), + } + } + + fn avg(operator: HavingOperator, right: Value, limit: u32) -> Self { + Self { + document_type_name: "review", + select: SelectProjection::avg("grade"), + having: clause(HavingAggregateFunction::Avg, "grade", operator, right), + order_ascending: None, + limit: Some(limit), + } + } + + fn ordered(mut self, ascending: bool) -> Self { + self.order_ascending = Some(ascending); + self + } + + fn order_by(&self) -> Vec { + match self.order_ascending { + None => Vec::new(), + Some(ascending) => { + let field = match self.select.field.as_str() { + "" => RANKED_COUNT_ORDER_KEY.to_string(), + field => field.to_string(), + }; + vec![OrderClause { field, ascending }] + } + } + } + } + + /// Run a case through the public dispatcher entry point — the same + /// call drive-abci's routing layer makes. + fn run( + drive: &Drive, + contract: &DataContract, + case: &HavingCase, + prove: bool, + ) -> Result { + let group_by = vec![GROUP_PROPERTY.to_string()]; + let having = vec![case.having.clone()]; + let order_by = case.order_by(); + let document_type = contract + .document_type_for_name(case.document_type_name) + .expect("doctype exists"); + drive.execute_document_having_request( + DocumentHavingRequest { + contract, + document_type, + group_by: &group_by, + select: case.select.clone(), + having: &having, + order_by: &order_by, + where_clauses: &[], + limit: case.limit, + offset: None, + has_start_at: false, + prove, + }, + None, + platform_version(), + ) + } + + fn entries_of(response: DocumentHavingResponse) -> Vec { + match response { + DocumentHavingResponse::Entries(entries) => entries, + DocumentHavingResponse::Proof(_) => panic!("expected entries, got a proof"), + } + } + + fn proof_of(response: DocumentHavingResponse) -> Vec { + match response { + DocumentHavingResponse::Proof(proof) => proof, + DocumentHavingResponse::Entries(_) => panic!("expected a proof, got entries"), + } + } + + fn keys_of(entries: &[RankedEntry]) -> Vec { + entries + .iter() + .map(|entry| { + String::from_utf8(entry.key.clone()).expect("fixture group keys are utf-8") + }) + .collect() + } + + /// Rebuild the query the way a client would: re-run the same + /// versioned validation (which resolves the bounds), then resolve + /// the index off the contract — the shape the SDK's proof helper + /// takes. + fn client_side_query<'a>( + contract: &'a DataContract, + case: &HavingCase, + ) -> DriveDocumentHavingQuery<'a> { + let group_by = vec![GROUP_PROPERTY.to_string()]; + let having = vec![case.having.clone()]; + let order_by = case.order_by(); + let mode = detect_having_mode( + &case.select, + &group_by, + &having, + &order_by, + &[], + RankedPaginationInputs { + limit: case.limit, + offset: None, + has_start_at: false, + }, + platform_version(), + ) + .expect("the case is well-formed"); + let indexes = contract + .document_types() + .get(case.document_type_name) + .expect("doctype exists") + .indexes(); + let index = find_ranked_index_for_axis( + indexes, + &mode.group_by_property, + mode.bounds.axis(), + &mode.aggregate_field, + ) + .expect("the fixture declares the axis"); + DriveDocumentHavingQuery { + document_type: contract + .document_type_for_name(case.document_type_name) + .expect("doctype exists"), + contract_id: contract.id_ref().to_buffer(), + document_type_name: case.document_type_name.to_string(), + index, + bounds: mode.bounds, + descending: mode.descending, + limit: mode.limit, + } + } + + fn grovedb_root_hash(drive: &Drive) -> [u8; 32] { + drive + .grove + .root_hash(None, &platform_version().drive.grove_version) + .unwrap() + .expect("root hash must be readable") + } + + /// Prove the case, verify the proof, and assert the verified + /// entries and root hash match the live database. + fn assert_proof_round_trips( + drive: &Drive, + contract: &DataContract, + case: &HavingCase, + expected: &[RankedEntry], + ) { + let proof = proof_of(run(drive, contract, case, true).expect("prove must succeed")); + let query = client_side_query(contract, case); + let (root_hash, verified) = query + .verify_having_range_proof(&proof, platform_version()) + .expect("the proof must verify"); + assert_eq!( + verified, expected, + "verified entries must equal what the unproven read returned" + ); + assert_eq!( + root_hash, + grovedb_root_hash(drive), + "the proof must reconstruct the live grovedb root hash" + ); + } + + /// Visits per restaurant: alpha 1, beta 3, gamma 2, delta 4. + /// `HAVING COUNT(*) > 2` must return exactly beta and delta, in + /// ascending count order (no ORDER BY), and the proof must commit + /// the same page. + #[test] + fn count_threshold_reads_and_proves_consistently() { + let (drive, contract) = setup_restaurants(); + insert_docs( + &drive, + &contract, + "visit", + "guests", + 100, + &[ + ("alpha", 1), + ("beta", 1), + ("beta", 2), + ("beta", 3), + ("gamma", 1), + ("gamma", 2), + ("delta", 1), + ("delta", 2), + ("delta", 3), + ("delta", 4), + ], + ); + + let case = HavingCase::count(HavingOperator::GreaterThan, Value::U64(2), 10); + let entries = entries_of(run(&drive, &contract, &case, false).expect("read succeeds")); + assert_eq!( + keys_of(&entries), + vec!["beta", "delta"], + "ascending count order: beta (3) before delta (4)" + ); + assert_eq!( + entries.iter().map(|e| e.value).collect::>(), + vec![RankedEntryValue::Count(3), RankedEntryValue::Count(4)] + ); + + assert_proof_round_trips(&drive, &contract, &case, &entries); + } + + /// Same state, descending: `HAVING $count >= 2 ORDER BY $count + /// DESC` walks from the largest matching count down. + #[test] + fn descending_walk_returns_biggest_matches_first() { + let (drive, contract) = setup_restaurants(); + insert_docs( + &drive, + &contract, + "visit", + "guests", + 200, + &[ + ("alpha", 1), + ("beta", 1), + ("beta", 2), + ("gamma", 1), + ("gamma", 2), + ("gamma", 3), + ], + ); + + let case = HavingCase::count(HavingOperator::GreaterThanOrEquals, Value::U64(2), 10) + .ordered(false); + let entries = entries_of(run(&drive, &contract, &case, false).expect("read succeeds")); + assert_eq!(keys_of(&entries), vec!["gamma", "beta"]); + + assert_proof_round_trips(&drive, &contract, &case, &entries); + } + + /// Tips per restaurant: alpha 150, beta 900, gamma 400. + /// `HAVING SUM(amount) BETWEEN 100 AND 500` returns alpha and gamma + /// — bounds inclusive on both ends. + #[test] + fn sum_between_reads_and_proves_consistently() { + let (drive, contract) = setup_restaurants(); + insert_docs( + &drive, + &contract, + "tip", + "amount", + 300, + &[("alpha", 100), ("alpha", 50), ("beta", 900), ("gamma", 400)], + ); + + let case = HavingCase::sum( + HavingOperator::Between, + Value::Array(vec![Value::I64(100), Value::I64(500)]), + 10, + ); + let entries = entries_of(run(&drive, &contract, &case, false).expect("read succeeds")); + assert_eq!(keys_of(&entries), vec!["alpha", "gamma"]); + assert_eq!( + entries.iter().map(|e| e.value).collect::>(), + vec![RankedEntryValue::Sum(150), RankedEntryValue::Sum(400)] + ); + + assert_proof_round_trips(&drive, &contract, &case, &entries); + } + + /// Reviews: alpha (90+80)/2 = 85, beta (60+70+50)/3 = 60, gamma 95. + /// `HAVING AVG(grade) >= 85` returns alpha and gamma; the entries + /// carry the exact fixed points. + #[test] + fn avg_threshold_reads_and_proves_consistently() { + let (drive, contract) = setup_restaurants(); + insert_docs( + &drive, + &contract, + "review", + "grade", + 400, + &[ + ("alpha", 90), + ("alpha", 80), + ("beta", 60), + ("beta", 70), + ("beta", 50), + ("gamma", 95), + ], + ); + + let case = HavingCase::avg(HavingOperator::GreaterThanOrEquals, Value::U64(85), 10); + let entries = entries_of(run(&drive, &contract, &case, false).expect("read succeeds")); + assert_eq!(keys_of(&entries), vec!["alpha", "gamma"]); + assert_eq!( + entries.iter().map(|e| e.value).collect::>(), + vec![ + RankedEntryValue::AvgFixedPoint(compute_avg_fixed_point(170, 2)), + RankedEntryValue::AvgFixedPoint(compute_avg_fixed_point(95, 1)), + ] + ); + + assert_proof_round_trips(&drive, &contract, &case, &entries); + } + + /// The limit cuts an over-long match set — and the cut page still + /// proves. With ascending order and `LIMIT 2`, the two *smallest* + /// matching counts come back. + #[test] + fn limit_cuts_the_match_set_and_the_cut_page_proves() { + let (drive, contract) = setup_restaurants(); + insert_docs( + &drive, + &contract, + "visit", + "guests", + 500, + &[ + ("alpha", 1), + ("alpha", 2), + ("beta", 1), + ("beta", 2), + ("beta", 3), + ("gamma", 1), + ("gamma", 2), + ("gamma", 3), + ("gamma", 4), + ], + ); + + let case = HavingCase::count(HavingOperator::GreaterThanOrEquals, Value::U64(2), 2); + let entries = entries_of(run(&drive, &contract, &case, false).expect("read succeeds")); + assert_eq!( + keys_of(&entries), + vec!["alpha", "beta"], + "three groups match but the limit keeps the two smallest" + ); + + assert_proof_round_trips(&drive, &contract, &case, &entries); + } + + /// A bound matching nothing is a legitimate, provable answer — + /// both against populated state and against a freshly registered + /// contract whose secondary is empty. + #[test] + fn an_empty_match_set_reads_empty_and_proves_empty() { + let (drive, contract) = setup_restaurants(); + + // Empty secondary (no documents at all): the unproven read + // returns the empty list, but grovedb's range prover — unlike + // the ranked surface's paginated prover — has no absence-proof + // shape for a completely empty tree and refuses. drive-abci + // maps this exact failure class onto an `InvalidArgument` + // telling the caller to retry unproved + // (`empty_ranking_proof_rejection`); at the drive level it + // surfaces as the grovedb error asserted here. If a future + // grovedb pin makes empty range proofs work, this arm should + // flip to a round-trip assertion. + let case = HavingCase::count(HavingOperator::GreaterThan, Value::U64(100), 10); + let entries = entries_of(run(&drive, &contract, &case, false).expect("read succeeds")); + assert!(entries.is_empty()); + let error = run(&drive, &contract, &case, true) + .expect_err("proving against an empty secondary is refused by grovedb"); + assert!( + format!("{error}").contains("Cannot create proof for empty tree"), + "the failure must be the recognized empty-tree class, got: {error}" + ); + + // Populated secondary, bound above every count: a genuine + // absence proof, which works — the tree has content to anchor + // the boundary commitments to. + insert_docs( + &drive, + &contract, + "visit", + "guests", + 600, + &[("alpha", 1), ("beta", 1), ("beta", 2)], + ); + let entries = entries_of(run(&drive, &contract, &case, false).expect("read succeeds")); + assert!(entries.is_empty()); + assert_proof_round_trips(&drive, &contract, &case, &entries); + } + + /// A proof generated for one bound must not verify as a different + /// bound **whose answer differs**: verification re-runs the Merk + /// query against the proof, so a wider bound demands proof of a + /// group the narrower proof never committed (gamma, count 2, below) + /// and fails. + /// + /// The state is chosen so the two bounds genuinely disagree. With + /// no group between the two thresholds the same proof *does* + /// verify under both — correctly, because the range boundaries + /// prove both claims — so the distinguishing group is the point of + /// the fixture. + #[test] + fn a_proof_does_not_verify_under_different_bounds() { + let (drive, contract) = setup_restaurants(); + insert_docs( + &drive, + &contract, + "visit", + "guests", + 700, + &[ + ("alpha", 1), + ("gamma", 1), + ("gamma", 2), + ("beta", 1), + ("beta", 2), + ("beta", 3), + ], + ); + + let over_two = HavingCase::count(HavingOperator::GreaterThan, Value::U64(2), 10); + let proof = proof_of(run(&drive, &contract, &over_two, true).expect("prove succeeds")); + + // Honest verification succeeds… + assert!(client_side_query(&contract, &over_two) + .verify_having_range_proof(&proof, platform_version()) + .is_ok()); + + // …but the same bytes under a different threshold must not. + let over_one = HavingCase::count(HavingOperator::GreaterThan, Value::U64(1), 10); + let mut tampered_query = client_side_query(&contract, &over_one); + assert_eq!( + tampered_query.bounds, + AxisRangeBounds::Count { + lo: 2, + hi: u64::MAX + } + ); + assert!( + tampered_query + .verify_having_range_proof(&proof, platform_version()) + .is_err(), + "a proof of `> 2` must not verify as `> 1`" + ); + + // Nor under a different direction or limit. + tampered_query = client_side_query(&contract, &over_two); + tampered_query.descending = true; + assert!(tampered_query + .verify_having_range_proof(&proof, platform_version()) + .is_err()); + + tampered_query = client_side_query(&contract, &over_two); + tampered_query.limit = 5; + assert!(tampered_query + .verify_having_range_proof(&proof, platform_version()) + .is_err()); + } + + /// A `having` on an axis no index declares is refused with the + /// contract keyword the author needs to add. The `review` doctype's + /// index is `rankedAverageable` only — a COUNT bound has no + /// covering secondary. + #[test] + fn a_bound_on_an_undeclared_axis_names_the_missing_keyword() { + let (drive, contract) = setup_restaurants(); + let case = HavingCase { + document_type_name: "review", + select: SelectProjection::count_star(), + having: clause( + HavingAggregateFunction::Count, + "", + HavingOperator::GreaterThan, + Value::U64(2), + ), + order_ascending: None, + limit: Some(10), + }; + let error = run(&drive, &contract, &case, false).expect_err("no covering axis"); + assert!( + format!("{error}").contains("rankedCountable"), + "the rejection must name the missing keyword, got: {error}" + ); + } + + /// Equal bounds are a point lookup on the axis: `HAVING SUM(amount) + /// = 400` returns exactly the group whose running sum is 400. + #[test] + fn equality_is_a_point_bound() { + let (drive, contract) = setup_restaurants(); + insert_docs( + &drive, + &contract, + "tip", + "amount", + 800, + &[("alpha", 150), ("beta", 400), ("gamma", 400)], + ); + + let case = HavingCase::sum(HavingOperator::Equal, Value::I64(400), 10); + let entries = entries_of(run(&drive, &contract, &case, false).expect("read succeeds")); + assert_eq!( + keys_of(&entries), + vec!["beta", "gamma"], + "equal sums tie-break by group key in walk direction" + ); + + assert_proof_round_trips(&drive, &contract, &case, &entries); + } + + /// Continuation-by-bound: after a page cut at the limit, the + /// caller tightens the bound past the last seen value and picks up + /// at the next *distinct* aggregate value. This is deliberately not + /// full pagination — a cut inside a tie cannot be continued (the + /// tied groups past the limit are unreachable without a + /// composite-key cursor); this fixture's counts are distinct, which + /// is the case the continuation serves. + #[test] + fn tightening_the_bound_continues_past_a_cut_page() { + let (drive, contract) = setup_restaurants(); + insert_docs( + &drive, + &contract, + "visit", + "guests", + 900, + &[ + ("alpha", 1), + ("alpha", 2), + ("beta", 1), + ("beta", 2), + ("beta", 3), + ("gamma", 1), + ("gamma", 2), + ("gamma", 3), + ("gamma", 4), + ], + ); + + // Page 1: counts >= 2, limit 1 → alpha (count 2). + let page_one = HavingCase::count(HavingOperator::GreaterThanOrEquals, Value::U64(2), 1); + let first = entries_of(run(&drive, &contract, &page_one, false).expect("read succeeds")); + assert_eq!(keys_of(&first), vec!["alpha"]); + let RankedEntryValue::Count(last_seen) = first[0].value else { + panic!("count axis returns count values"); + }; + + // Page 2: counts > last seen → beta, gamma. + let page_two = HavingCase::count(HavingOperator::GreaterThan, Value::U64(last_seen), 10); + let rest = entries_of(run(&drive, &contract, &page_two, false).expect("read succeeds")); + assert_eq!(keys_of(&rest), vec!["beta", "gamma"]); + + assert_proof_round_trips(&drive, &contract, &page_two, &rest); + } +} + +mod identifier_group_keys { + //! `SELECT AVG(grade) FROM grades GROUP BY identityId HAVING + //! AVG(grade) > 80` — the same surface as the `execution` suite + //! above, but with a **32-byte identifier** as the group key + //! instead of a string. Identifier and string properties encode + //! differently into the axis secondary's `sort_key‖group_key` + //! keyspace, so this pins that identifier group keys round-trip + //! byte-exact through the read, the proof, and the verifier. + + use super::super::drive_dispatcher::{DocumentHavingRequest, DocumentHavingResponse}; + use super::super::mode_detection::detect_having_mode; + use super::super::DriveDocumentHavingQuery; + use super::clause; + use crate::drive::Drive; + use crate::query::drive_document_ranked_query::index_picker::find_ranked_index_for_axis; + use crate::query::drive_document_ranked_query::{ + RankedEntry, RankedEntryValue, RankedPaginationInputs, + }; + use crate::query::having::{HavingAggregateFunction, HavingOperator}; + use crate::query::projection::SelectProjection; + use crate::query::OrderClause; + use crate::util::object_size_info::DocumentInfo::DocumentRefInfo; + use crate::util::object_size_info::{DocumentAndContractInfo, OwnedDocumentInfo}; + use crate::util::storage_flags::StorageFlags; + use crate::util::test_helpers::setup::setup_drive_with_initial_state_structure; + use dpp::block::block_info::BlockInfo; + use dpp::data_contract::accessors::v0::DataContractV0Getters; + use dpp::data_contract::document_type::accessors::DocumentTypeV0Getters; + use dpp::data_contract::document_type::random_document::CreateRandomDocument; + use dpp::document::{Document, DocumentV0Setters}; + use dpp::platform_value::Value; + use dpp::prelude::DataContract; + use dpp::tests::json_document::json_document_to_contract; + use dpp::version::PlatformVersion; + use grovedb::element::indexed::compute_avg_fixed_point; + use std::collections::BTreeMap; + + const GROUP_PROPERTY: &str = "identityId"; + const DOCUMENT_TYPE: &str = "grade"; + + fn platform_version() -> &'static PlatformVersion { + PlatformVersion::latest() + } + + fn setup_grades_ranked() -> (Drive, DataContract) { + let drive = setup_drive_with_initial_state_structure(None); + let pv = platform_version(); + let contract = json_document_to_contract( + "tests/supporting_files/contract/grades/grades-ranked-contract.json", + false, + pv, + ) + .expect("expected to parse the ranked grades contract"); + drive + .apply_contract( + &contract, + BlockInfo::default(), + true, + StorageFlags::optional_default_as_cow(), + None, + pv, + ) + .expect("expected to apply the ranked grades contract"); + (drive, contract) + } + + fn insert_grades( + drive: &Drive, + contract: &DataContract, + first_seed: u64, + rows: &[([u8; 32], i64)], + ) { + let pv = platform_version(); + let document_type = contract + .document_type_for_name(DOCUMENT_TYPE) + .expect("grade doctype exists"); + for (i, (identity, grade)) in rows.iter().enumerate() { + let mut doc: Document = document_type + .random_document(Some(first_seed + i as u64), pv) + .expect("random document"); + let mut props = BTreeMap::new(); + props.insert(GROUP_PROPERTY.to_string(), Value::Identifier(*identity)); + props.insert("grade".to_string(), Value::I64(*grade)); + doc.set_properties(props); + drive + .add_document_for_contract( + DocumentAndContractInfo { + owned_document_info: OwnedDocumentInfo { + document_info: DocumentRefInfo((&doc, None)), + owner_id: None, + }, + contract, + document_type, + }, + false, + BlockInfo::default(), + true, + None, + pv, + None, + ) + .expect("expected to insert a grade document"); + } + } + + fn run( + drive: &Drive, + contract: &DataContract, + order_by: &[OrderClause], + prove: bool, + ) -> DocumentHavingResponse { + let group_by = vec![GROUP_PROPERTY.to_string()]; + let having = vec![clause( + HavingAggregateFunction::Avg, + "grade", + HavingOperator::GreaterThan, + Value::U64(80), + )]; + drive + .execute_document_having_request( + DocumentHavingRequest { + contract, + document_type: contract + .document_type_for_name(DOCUMENT_TYPE) + .expect("grade doctype exists"), + group_by: &group_by, + select: SelectProjection::avg("grade"), + having: &having, + order_by, + where_clauses: &[], + limit: Some(10), + offset: None, + has_start_at: false, + prove, + }, + None, + platform_version(), + ) + .expect("the having request must execute") + } + + fn client_side_query<'a>( + contract: &'a DataContract, + order_by: &[OrderClause], + ) -> DriveDocumentHavingQuery<'a> { + let group_by = vec![GROUP_PROPERTY.to_string()]; + let having = vec![clause( + HavingAggregateFunction::Avg, + "grade", + HavingOperator::GreaterThan, + Value::U64(80), + )]; + let mode = detect_having_mode( + &SelectProjection::avg("grade"), + &group_by, + &having, + order_by, + &[], + RankedPaginationInputs { + limit: Some(10), + offset: None, + has_start_at: false, + }, + platform_version(), + ) + .expect("the case is well-formed"); + let indexes = contract + .document_types() + .get(DOCUMENT_TYPE) + .expect("grade doctype exists") + .indexes(); + let index = find_ranked_index_for_axis( + indexes, + &mode.group_by_property, + mode.bounds.axis(), + &mode.aggregate_field, + ) + .expect("the fixture declares the avg axis"); + DriveDocumentHavingQuery { + document_type: contract + .document_type_for_name(DOCUMENT_TYPE) + .expect("grade doctype exists"), + contract_id: contract.id_ref().to_buffer(), + document_type_name: DOCUMENT_TYPE.to_string(), + index, + bounds: mode.bounds, + descending: mode.descending, + limit: mode.limit, + } + } + + fn assert_proof_round_trips( + drive: &Drive, + contract: &DataContract, + order_by: &[OrderClause], + expected: &[RankedEntry], + ) { + let proof = match run(drive, contract, order_by, true) { + DocumentHavingResponse::Proof(proof) => proof, + DocumentHavingResponse::Entries(_) => panic!("expected a proof, got entries"), + }; + let (root_hash, verified) = client_side_query(contract, order_by) + .verify_having_range_proof(&proof, platform_version()) + .expect("the proof must verify"); + assert_eq!( + verified, expected, + "verified entries must equal what the unproven read returned" + ); + assert_eq!( + root_hash, + drive + .grove + .root_hash(None, &platform_version().drive.grove_version) + .unwrap() + .expect("root hash must be readable"), + "the proof must reconstruct the live grovedb root hash" + ); + } + + /// Averages exactly *at* the threshold stay out (`>` is strict), + /// fractional averages just above it come in (80.5 > 80 even + /// though both grades round-trip as integers), and the entry keys + /// are the raw 32-byte identifiers. + #[test] + fn avg_threshold_over_identifier_groups_reads_and_proves() { + let (drive, contract) = setup_grades_ranked(); + let at_threshold = [1u8; 32]; // 80, 80 → avg 80: excluded + let just_above = [2u8; 32]; // 80, 81 → avg 80.5: included + let well_above = [3u8; 32]; // 85, 95 → avg 90: included + let below = [4u8; 32]; // 60, 80 → avg 70: excluded + insert_grades( + &drive, + &contract, + 1000, + &[ + (at_threshold, 80), + (at_threshold, 80), + (just_above, 80), + (just_above, 81), + (well_above, 85), + (well_above, 95), + (below, 60), + (below, 80), + ], + ); + + let entries = match run(&drive, &contract, &[], false) { + DocumentHavingResponse::Entries(entries) => entries, + DocumentHavingResponse::Proof(_) => panic!("expected entries, got a proof"), + }; + assert_eq!( + entries, + vec![ + RankedEntry { + key: just_above.to_vec(), + value: RankedEntryValue::AvgFixedPoint(compute_avg_fixed_point(161, 2)), + }, + RankedEntry { + key: well_above.to_vec(), + value: RankedEntryValue::AvgFixedPoint(compute_avg_fixed_point(180, 2)), + }, + ], + "ascending walk: 80.5 then 90, keyed by raw identifier bytes" + ); + assert_proof_round_trips(&drive, &contract, &[], &entries); + + // `ORDER BY AVG(grade) DESC` walks the same match set from the + // top; the identifier keys must survive the flipped direction + // and its proof too. + let descending = vec![OrderClause { + field: "grade".to_string(), + ascending: false, + }]; + let flipped = match run(&drive, &contract, &descending, false) { + DocumentHavingResponse::Entries(entries) => entries, + DocumentHavingResponse::Proof(_) => panic!("expected entries, got a proof"), + }; + assert_eq!( + flipped.iter().map(|e| &e.key).collect::>(), + vec![&well_above.to_vec(), &just_above.to_vec()], + "descending walk: 90 then 80.5" + ); + assert_proof_round_trips(&drive, &contract, &descending, &flipped); + } +} diff --git a/packages/rs-drive/src/query/drive_document_ranked_query/mode_detection/mod.rs b/packages/rs-drive/src/query/drive_document_ranked_query/mode_detection/mod.rs new file mode 100644 index 00000000000..b661ad7cfac --- /dev/null +++ b/packages/rs-drive/src/query/drive_document_ranked_query/mode_detection/mod.rs @@ -0,0 +1,91 @@ +//! Request-shape validation for the ranked query, and the versioned +//! `(select, group_by, order_by, limit, offset)` → [`DocumentRankedMode`] +//! resolution. +//! +//! Pure functions on the request shape — no Drive, no contract, no +//! indexes. Available under `server` (the dispatcher validates before +//! executing) and `verify` (the SDK validates the same way before +//! attempting proof verification), so both sides agree on which requests +//! are well-formed and on the `(axis, descending, k, offset)` tuple a +//! well-formed one resolves to. Index-dependent validation ("does an +//! index actually cover this axis?") needs the document type's index map +//! and lives in [`super::index_picker`]. +//! +//! Versioned through +//! `platform_version.drive.methods.document.query.detect_ranked_mode`, +//! the same way +//! [`DriveDocumentCountQuery::detect_mode_versioned`](super::super::drive_document_count_query::DriveDocumentCountQuery::detect_mode_versioned) +//! routes count's table: the accepted request grammar is a consensus- +//! adjacent contract on the query surface, so relaxing it later has to +//! land behind a method-version bump rather than changing what an +//! already-deployed protocol version accepts. + +use super::{ + DocumentRankedMode, RankedAxis, RankedPaginationInputs, MAX_RANKED_LIMIT, + RANKED_COUNT_ORDER_KEY, +}; +use crate::error::query::QuerySyntaxError; +use crate::error::Error; +use crate::query::having::HavingClause; +use crate::query::projection::{SelectFunction, SelectProjection}; +use crate::query::{OrderClause, WhereClause}; +use dpp::version::PlatformVersion; + +/// Versioned entry point. Routes through +/// `platform_version.drive.methods.document.query.detect_ranked_mode`; +/// today only `0` is defined and maps to [`detect_ranked_mode_v0`] +/// verbatim. +pub fn detect_ranked_mode( + select: &SelectProjection, + group_by: &[String], + having: &[HavingClause], + order_by: &[OrderClause], + where_clauses: &[WhereClause], + pagination: RankedPaginationInputs, + platform_version: &PlatformVersion, +) -> Result { + match platform_version + .drive + .methods + .document + .query + .detect_ranked_mode + { + 0 => detect_ranked_mode_v0( + select, + group_by, + having, + order_by, + where_clauses, + pagination, + ), + version => Err(Error::Query(QuerySyntaxError::Unsupported(format!( + "detect_ranked_mode: unknown method version {version}; only 0 is supported" + )))), + } +} + +/// The `ORDER BY` field name that names a given select's aggregate. +/// +/// `SUM(f)` / `AVG(f)` are ordered by naming `f` — the same field the +/// projection aggregates, which is how SQL's `ORDER BY avg(grade)` +/// reads once the aggregate function is already fixed by the `SELECT`. +/// `COUNT(*)` has no field, so it is named by the +/// [`RANKED_COUNT_ORDER_KEY`] sentinel. +/// +/// Public because request *builders* need it as much as the validator +/// does: an SDK offering `.order_by_selected_aggregate(…)` has to emit +/// the same string this function expects to read back, and a second +/// copy of the sentinel rule is a silent-rejection bug waiting for the +/// first `COUNT(*)` ranking. +pub fn ranked_order_key(select: &SelectProjection) -> &str { + match select.function { + SelectFunction::Count if select.field.is_empty() => RANKED_COUNT_ORDER_KEY, + _ => select.field.as_str(), + } +} + +mod v0; +// Re-exported so the dispatcher's callers (`drive_dispatcher`, the +// test suites) keep addressing the frozen grammar by its old path. +pub use v0::detect_ranked_mode_v0; diff --git a/packages/rs-drive/src/query/drive_document_ranked_query/mode_detection.rs b/packages/rs-drive/src/query/drive_document_ranked_query/mode_detection/v0/mod.rs similarity index 79% rename from packages/rs-drive/src/query/drive_document_ranked_query/mode_detection.rs rename to packages/rs-drive/src/query/drive_document_ranked_query/mode_detection/v0/mod.rs index f42de99cfe8..6dfa811229c 100644 --- a/packages/rs-drive/src/query/drive_document_ranked_query/mode_detection.rs +++ b/packages/rs-drive/src/query/drive_document_ranked_query/mode_detection/v0/mod.rs @@ -1,25 +1,9 @@ -//! Request-shape validation for the ranked query, and the versioned -//! `(select, group_by, order_by, limit, offset)` → [`DocumentRankedMode`] -//! resolution. -//! -//! Pure functions on the request shape — no Drive, no contract, no -//! indexes. Available under `server` (the dispatcher validates before -//! executing) and `verify` (the SDK validates the same way before -//! attempting proof verification), so both sides agree on which requests -//! are well-formed and on the `(axis, descending, k, offset)` tuple a -//! well-formed one resolves to. Index-dependent validation ("does an -//! index actually cover this axis?") needs the document type's index map -//! and lives in [`super::index_picker`]. -//! -//! Versioned through -//! `platform_version.drive.methods.document.query.detect_ranked_mode`, -//! the same way -//! [`DriveDocumentCountQuery::detect_mode_versioned`](super::super::drive_document_count_query::DriveDocumentCountQuery::detect_mode_versioned) -//! routes count's table: the accepted request grammar is a consensus- -//! adjacent contract on the query surface, so relaxing it later has to -//! land behind a method-version bump rather than changing what an -//! already-deployed protocol version accepts. +//! Feature version 0 of the grammar — the frozen implementation +//! behind the `mode_detection` dispatcher. Everything private in +//! this file is a v0 internal: a later grammar version gets its own +//! `vN/` sibling rather than editing this one. +use super::ranked_order_key; use super::{ DocumentRankedMode, RankedAxis, RankedPaginationInputs, MAX_RANKED_LIMIT, RANKED_COUNT_ORDER_KEY, @@ -29,61 +13,6 @@ use crate::error::Error; use crate::query::having::HavingClause; use crate::query::projection::{SelectFunction, SelectProjection}; use crate::query::{OrderClause, WhereClause}; -use dpp::version::PlatformVersion; - -/// Versioned entry point. Routes through -/// `platform_version.drive.methods.document.query.detect_ranked_mode`; -/// today only `0` is defined and maps to [`detect_ranked_mode_v0`] -/// verbatim. -pub fn detect_ranked_mode( - select: &SelectProjection, - group_by: &[String], - having: &[HavingClause], - order_by: &[OrderClause], - where_clauses: &[WhereClause], - pagination: RankedPaginationInputs, - platform_version: &PlatformVersion, -) -> Result { - match platform_version - .drive - .methods - .document - .query - .detect_ranked_mode - { - 0 => detect_ranked_mode_v0( - select, - group_by, - having, - order_by, - where_clauses, - pagination, - ), - version => Err(Error::Query(QuerySyntaxError::Unsupported(format!( - "detect_ranked_mode: unknown method version {version}; only 0 is supported" - )))), - } -} - -/// The `ORDER BY` field name that names a given select's aggregate. -/// -/// `SUM(f)` / `AVG(f)` are ordered by naming `f` — the same field the -/// projection aggregates, which is how SQL's `ORDER BY avg(grade)` -/// reads once the aggregate function is already fixed by the `SELECT`. -/// `COUNT(*)` has no field, so it is named by the -/// [`RANKED_COUNT_ORDER_KEY`] sentinel. -/// -/// Public because request *builders* need it as much as the validator -/// does: an SDK offering `.order_by_selected_aggregate(…)` has to emit -/// the same string this function expects to read back, and a second -/// copy of the sentinel rule is a silent-rejection bug waiting for the -/// first `COUNT(*)` ranking. -pub fn ranked_order_key(select: &SelectProjection) -> &str { - match select.function { - SelectFunction::Count if select.field.is_empty() => RANKED_COUNT_ORDER_KEY, - _ => select.field.as_str(), - } -} /// v0 of the ranked request grammar. /// diff --git a/packages/rs-drive/src/query/drive_document_ranked_query/path.rs b/packages/rs-drive/src/query/drive_document_ranked_query/path.rs index be504f18ea2..a66c13b08c6 100644 --- a/packages/rs-drive/src/query/drive_document_ranked_query/path.rs +++ b/packages/rs-drive/src/query/drive_document_ranked_query/path.rs @@ -17,6 +17,35 @@ use super::DriveDocumentRankedQuery; use crate::drive::RootTree; use crate::error::drive::DriveError; use crate::error::Error; +use dpp::data_contract::document_type::Index; + +/// Path of a single-property index's terminal property-name tree — +/// shared by the ranked and having-range query surfaces, which read +/// the same indexed tree. See +/// [`DriveDocumentRankedQuery::indexed_property_name_tree_path`] for +/// the segment layout and the single-property requirement. +pub(crate) fn indexed_property_name_tree_path_for_index( + contract_id: &[u8; 32], + document_type_name: &str, + index: &Index, +) -> Result>, Error> { + let [property] = index.properties.as_slice() else { + return Err(Error::Drive(DriveError::NotSupported( + "ranked and having-range queries require a single-property index: the \ + axis secondary lives on the index's terminal property-name tree, and \ + for a compound index that tree sits under a prefix value tree whose \ + value only a `where` clause could name — but these queries accept no \ + `where` clauses", + ))); + }; + Ok(vec![ + vec![RootTree::DataContractDocuments as u8], + contract_id.to_vec(), + vec![1u8], + document_type_name.as_bytes().to_vec(), + property.name.as_bytes().to_vec(), + ]) +} impl DriveDocumentRankedQuery<'_> { /// Path of the **terminal property-name tree** — the indexed tree @@ -44,21 +73,10 @@ impl DriveDocumentRankedQuery<'_> { /// typed error than a path pointing at a prefix level whose element /// is not an indexed tree at all. pub fn indexed_property_name_tree_path(&self) -> Result>, Error> { - let [property] = self.index.properties.as_slice() else { - return Err(Error::Drive(DriveError::NotSupported( - "ranked queries require a single-property index: the ranked secondary \ - lives on the index's terminal property-name tree, and for a compound \ - index that tree sits under a prefix value tree whose value only a \ - `where` clause could name — but ranked queries accept no `where` \ - clauses", - ))); - }; - Ok(vec![ - vec![RootTree::DataContractDocuments as u8], - self.contract_id.to_vec(), - vec![1u8], - self.document_type_name.as_bytes().to_vec(), - property.name.as_bytes().to_vec(), - ]) + indexed_property_name_tree_path_for_index( + &self.contract_id, + &self.document_type_name, + self.index, + ) } } diff --git a/packages/rs-drive/src/query/having.rs b/packages/rs-drive/src/query/having.rs index c82a3015d76..c861db287d7 100644 --- a/packages/rs-drive/src/query/having.rs +++ b/packages/rs-drive/src/query/having.rs @@ -32,9 +32,21 @@ //! and the SDK's request builder //! (`rs-sdk/src/platform/documents/document_query.rs`) so the //! drive-side struct is the single source of truth for the shape. -//! **No part of this grammar executes today**: every non-empty -//! `having` is rejected with `QuerySyntaxError::Unsupported`. The -//! types exist so the wire surface is stable as evaluation lands. +//! +//! **What executes (protocol version 14+)**: a grouped aggregate +//! carrying exactly one clause that bounds the aggregate the select +//! projects, with a contiguous-range operator (`=`, `>`, `>=`, `<`, +//! `<=`, the four `BETWEEN*` variants). It is served as a +//! value-bounded range read of the covering ranked index's axis +//! secondary — see `drive_document_having_query::mode_detection` for +//! the versioned grammar. Everything else the types can express +//! remains rejected with `QuerySyntaxError`: multiple clauses +//! (implicit AND would need a per-candidate post-check no executor +//! performs), a clause on a different aggregate than the select's, +//! the non-contiguous operators (`!=`, `IN`), and `having` without +//! `group_by`. Protocol version 13 and earlier reject every +//! non-empty `having`, so mixed-version networks agree across the +//! upgrade. use dpp::platform_value::Value; #[cfg(feature = "serde")] diff --git a/packages/rs-drive/src/query/mod.rs b/packages/rs-drive/src/query/mod.rs index 53fad48e77c..a601d420761 100644 --- a/packages/rs-drive/src/query/mod.rs +++ b/packages/rs-drive/src/query/mod.rs @@ -17,6 +17,14 @@ pub use { drive_document_count_query::{ CountMode, DocumentCountMode, DriveDocumentCountQuery, SplitCountEntry, }, + // Having-range verifier-shareable types — same split as ranked: + // `DocumentHavingMode` + `AxisRangeBounds` to re-run the same + // versioned request validation (and bounds translation) the prover + // ran, `DriveDocumentHavingQuery` to rebuild the proved grove path + // and secondary query. Entries reuse the ranked `RankedEntry` shape. + drive_document_having_query::{ + AxisRangeBounds, DocumentHavingMode, DriveDocumentHavingQuery, MAX_HAVING_LIMIT, + }, // Ranked-query verifier-shareable types. The verifier needs the // whole set: `DocumentRankedMode` + `RankedPaginationInputs` to // re-run the same versioned request validation the prover ran, @@ -71,6 +79,13 @@ pub use drive_document_average_query::{DocumentAverageRequest, DocumentAverageRe // as the count / sum / average request types above. #[cfg(feature = "server")] pub use drive_document_ranked_query::{DocumentRankedRequest, DocumentRankedResponse}; + +// `DocumentHavingRequest` / `DocumentHavingResponse` are the +// server-side dispatcher ABI for the having-range surface — the types +// drive-abci's routing layer names. Server-only for the same reason as +// the ranked request types above. +#[cfg(feature = "server")] +pub use drive_document_having_query::{DocumentHavingRequest, DocumentHavingResponse}; // Imports available when either "server" or "verify" features are enabled #[cfg(any(feature = "server", feature = "verify"))] use { @@ -243,6 +258,14 @@ pub mod drive_document_sum_query; #[cfg(any(feature = "server", feature = "verify"))] pub mod drive_document_average_query; +/// A query to filter an index's groups by a per-group aggregate bound — +/// "hashtags with more than 100 posts" — served as a value-bounded +/// range read of the same per-axis secondary Merk the ranked surface +/// walks (PR #657, PV14). Like ranked, it never opens the value trees, +/// so a having-range read is `O(log n + k)` with a proof. +#[cfg(any(feature = "server", feature = "verify"))] +pub mod drive_document_having_query; + /// A query to rank an index's groups by a per-group aggregate — "top /// 5 restaurants by average grade" — reading grovedb's per-axis /// secondary Merk of an indexed tree (PR #657, PV14). Unlike the diff --git a/packages/rs-drive/src/verify/document_having/mod.rs b/packages/rs-drive/src/verify/document_having/mod.rs new file mode 100644 index 00000000000..a3ddcec97e1 --- /dev/null +++ b/packages/rs-drive/src/verify/document_having/mod.rs @@ -0,0 +1,19 @@ +//! Verifies grovedb proofs produced by the having-range +//! (`GROUP BY … HAVING LIMIT n`) query surface. +//! +//! Mirrors the layering of [`super::document_ranked`]: a pure +//! grovedb-level verifier as a method on +//! [`DriveDocumentHavingQuery`](crate::query::DriveDocumentHavingQuery) +//! taking raw `proof: &[u8]` and returning `(RootHash, T)`. The +//! tenderdash signature composition that wraps this call lives in +//! `rs-drive-proof-verifier`. +//! +//! Only one verifier exists here, for the same reason as on the ranked +//! surface: every having-range request — any of the three axes, either +//! direction, any contiguous bound — resolves to one +//! `prove_indexed_axis_query` envelope that differs only in the Merk +//! query (the encoded bounds + direction) and limit it echoes. + +/// Indexed-axis range proof verification — returns the groups the proof +/// commits to as falling inside the bound, in axis order. +pub mod verify_having_range_proof; diff --git a/packages/rs-drive/src/verify/document_having/verify_having_range_proof/mod.rs b/packages/rs-drive/src/verify/document_having/verify_having_range_proof/mod.rs new file mode 100644 index 00000000000..b4ba63ead28 --- /dev/null +++ b/packages/rs-drive/src/verify/document_having/verify_having_range_proof/mod.rs @@ -0,0 +1,52 @@ +mod v0; + +use crate::error::drive::DriveError; +use crate::error::Error; +use crate::query::DriveDocumentHavingQuery; +use crate::verify::RootHash; +use dpp::version::PlatformVersion; + +impl DriveDocumentHavingQuery<'_> { + /// Verifies a grovedb indexed-axis range proof and returns + /// `(root_hash, entries)`. + /// + /// Counterpart to the prover-side + /// [`execute_range_with_proof`](Self::execute_range_with_proof). + /// Both sides derive the proved subtree from + /// [`indexed_property_name_tree_path`](Self::indexed_property_name_tree_path) + /// and the secondary query from + /// [`AxisRangeBounds::merk_query`](crate::query::drive_document_having_query::AxisRangeBounds::merk_query), + /// so the verifier cannot drift from the prover on *which* bound over + /// *which* tree it is checking. + /// + /// The returned entries are in axis order in the walk direction, + /// exactly as the unproven + /// [`execute_range_no_proof`](Self::execute_range_no_proof) would + /// return them. The caller combines `root_hash` with the surrounding + /// tenderdash signature — see `rs-drive-proof-verifier` for the + /// canonical composition. + /// + /// # Arguments + /// * `proof` — raw grovedb proof bytes. + /// * `platform_version` — selects the method version. + pub fn verify_having_range_proof( + &self, + proof: &[u8], + platform_version: &PlatformVersion, + ) -> Result<(RootHash, Vec), Error> { + match platform_version + .drive + .methods + .verify + .document_ranked + .verify_having_range_proof + { + 0 => self.verify_having_range_proof_v0(proof), + version => Err(Error::Drive(DriveError::UnknownVersionMismatch { + method: "DriveDocumentHavingQuery::verify_having_range_proof".to_string(), + known_versions: vec![0], + received: version, + })), + } + } +} diff --git a/packages/rs-drive/src/verify/document_having/verify_having_range_proof/v0/mod.rs b/packages/rs-drive/src/verify/document_having/verify_having_range_proof/v0/mod.rs new file mode 100644 index 00000000000..fc9a0ab1727 --- /dev/null +++ b/packages/rs-drive/src/verify/document_having/verify_having_range_proof/v0/mod.rs @@ -0,0 +1,112 @@ +use crate::error::drive::DriveError; +use crate::error::Error; +use crate::query::{DriveDocumentHavingQuery, RankedAxis, RankedEntry, RankedEntryValue}; +use crate::verify::RootHash; +use grovedb::operations::proof::indexed_axis::AxisEntries; +use grovedb::GroveDb; + +impl DriveDocumentHavingQuery<'_> { + /// v0 of [`Self::verify_having_range_proof`]. + /// + /// Rebuilds the proved subtree path with + /// [`Self::indexed_property_name_tree_path`] and the secondary query + /// with + /// [`AxisRangeBounds::merk_query`](crate::query::drive_document_having_query::AxisRangeBounds::merk_query), + /// then hands the proof to the matching + /// `GroveDb::verify_indexed_*_query` — an associated function, no + /// database handle, so this compiles and runs in a verifier-only + /// build. + /// + /// Three things are checked before the entries are returned: + /// + /// 1. **The envelope matches this query.** grovedb re-checks the + /// proof against the reconstructed Merk query (the encoded bounds + /// and walk direction) and the expected limit, so a proof + /// generated for a different bound — or a different direction, or + /// a different limit — is rejected rather than silently + /// reinterpreted. Completeness rides on the same check: a Merk + /// range proof commits its boundaries, so an in-range group the + /// prover omitted fails reconstruction. + /// 2. **The result's axis shape matches the requested axis** — the + /// same belt-and-braces check the ranked verifier does. + /// 3. **At most `limit` entries.** Fewer is normal — fewer groups + /// may match the bound — but more would mean the proof committed + /// a longer walk than the request authorized. + /// + /// No `platform_version` argument: the parent dispatcher already + /// consumed it to select this version, and verification derives + /// everything else from the proof bytes plus the query. + #[inline(always)] + pub(super) fn verify_having_range_proof_v0( + &self, + proof: &[u8], + ) -> Result<(RootHash, Vec), Error> { + let path = self.indexed_property_name_tree_path()?; + let path_refs: Vec<&[u8]> = path.iter().map(|segment| segment.as_slice()).collect(); + let secondary_query = self.bounds.merk_query(self.descending); + + let result = match self.bounds.axis() { + RankedAxis::Count => GroveDb::verify_indexed_count_query( + proof, + path_refs.as_slice(), + secondary_query, + Some(self.limit), + ), + RankedAxis::Sum => GroveDb::verify_indexed_sum_query( + proof, + path_refs.as_slice(), + secondary_query, + Some(self.limit), + ), + RankedAxis::Avg => GroveDb::verify_indexed_avg_query( + proof, + path_refs.as_slice(), + secondary_query, + Some(self.limit), + ), + } + .map_err(|e| Error::GroveDB(Box::new(e)))?; + + let entries = match (self.bounds.axis(), result.entries) { + (RankedAxis::Count, AxisEntries::Count(entries)) => entries + .into_iter() + .map(|(count, key)| RankedEntry { + key, + value: RankedEntryValue::Count(count), + }) + .collect::>(), + (RankedAxis::Sum, AxisEntries::Sum(entries)) => entries + .into_iter() + .map(|(sum, key)| RankedEntry { + key, + value: RankedEntryValue::Sum(sum), + }) + .collect::>(), + (RankedAxis::Avg, AxisEntries::Avg(entries)) => entries + .into_iter() + .map(|(avg, key)| RankedEntry { + key, + value: RankedEntryValue::AvgFixedPoint(avg), + }) + .collect::>(), + (axis, other) => { + return Err(Error::Drive(DriveError::CorruptedDriveState(format!( + "having range proof for the {axis:?} axis verified to {} entries of a \ + different axis shape", + other.len() + )))); + } + }; + + if entries.len() > self.limit as usize { + return Err(Error::Drive(DriveError::CorruptedDriveState(format!( + "having range proof for the {:?} axis verified to {} entries for limit = {}", + self.bounds.axis(), + entries.len(), + self.limit + )))); + } + + Ok((result.root_hash, entries)) + } +} diff --git a/packages/rs-drive/src/verify/mod.rs b/packages/rs-drive/src/verify/mod.rs index 8777f4b8397..3875f2b8507 100644 --- a/packages/rs-drive/src/verify/mod.rs +++ b/packages/rs-drive/src/verify/mod.rs @@ -7,6 +7,10 @@ pub mod document; /// Document-count verification methods on proofs (the /// `GetDocumentsCount` endpoint's prove-path verifiers). pub mod document_count; +/// Having-range verification methods on proofs (the +/// `GROUP BY … HAVING LIMIT n` surface's +/// prove-path verifier). +pub mod document_having; /// Document-ranked verification methods on proofs (the /// `GROUP BY … ORDER BY LIMIT n` surface's prove-path /// verifier). diff --git a/packages/rs-drive/tests/supporting_files/contract/grades/grades-ranked-contract.json b/packages/rs-drive/tests/supporting_files/contract/grades/grades-ranked-contract.json new file mode 100644 index 00000000000..0a51abcedf7 --- /dev/null +++ b/packages/rs-drive/tests/supporting_files/contract/grades/grades-ranked-contract.json @@ -0,0 +1,45 @@ +{ + "$formatVersion": "0", + "id": "9gradesS5w7Y9R4nDqJk2vHpL3uM6tF1xE8cA2bN7zXq", + "ownerId": "7m6mTfWqkrCnvLLPK3eqxQM2x2RDpYV6dsAyhVKsAEAQ", + "version": 1, + "documentSchemas": { + "grade": { + "type": "object", + "documentsMutable": false, + "canBeDeleted": false, + "indices": [ + { + "name": "byIdentity", + "properties": [ + { "identityId": "asc" } + ], + "averageable": "grade", + "rangeAverageable": true, + "rankedAverageable": true + } + ], + "properties": { + "identityId": { + "type": "array", + "byteArray": true, + "minItems": 32, + "maxItems": 32, + "position": 0, + "contentMediaType": "application/x.dash.dpp.identifier" + }, + "grade": { + "type": "integer", + "minimum": 0, + "maximum": 100, + "position": 1 + } + }, + "required": [ + "identityId", + "grade" + ], + "additionalProperties": false + } + } +} diff --git a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_query_versions/mod.rs b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_query_versions/mod.rs index 7c3e3e6dab1..b0594decefc 100644 --- a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_query_versions/mod.rs +++ b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_query_versions/mod.rs @@ -1,6 +1,7 @@ pub mod v0; pub mod v1; pub mod v2; +pub mod v3; use versioned_feature_core::{FeatureVersion, FeatureVersionBounds}; diff --git a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_query_versions/v3.rs b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_query_versions/v3.rs new file mode 100644 index 00000000000..74c2e14564a --- /dev/null +++ b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_query_versions/v3.rs @@ -0,0 +1,31 @@ +use crate::version::drive_abci_versions::drive_abci_query_versions::v2::DRIVE_ABCI_QUERY_VERSIONS_V2; +use crate::version::drive_abci_versions::drive_abci_query_versions::{ + DriveAbciDocumentQueryHelperVersions, DriveAbciQueryVersions, +}; + +/// Version 3 of the Drive ABCI query versions. +/// +/// Differs from v2 in exactly one slot: +/// `document_query_helpers.compute_aggregate_mode_and_check_limit` is 2 +/// rather than 1. That is the boolean-`HAVING` routing gate. The v1 +/// helper rejects every non-empty `having` ("HAVING clause is not yet +/// implemented"); the v2 helper routes a grouped aggregate carrying +/// exactly one `having` clause (`GROUP BY p HAVING +/// LIMIT n`) to the having-range executor, which serves it as a +/// value-bounded range read of the covering ranked index's axis +/// secondary. Everything else — including multi-clause `having` and +/// `having` on a select with no ranked axis — keeps the v1 behavior. +/// +/// Same mixed-network rationale as the v1 → v2 flip: earlier protocol +/// versions keep the v2 table and keep rejecting the shape, so nodes +/// agree until the upgrade carries. The wire surface is unchanged — +/// `GetDocumentsRequestV1.having` has been wire-stable since the v1 +/// document query, and the response reuses the additive +/// `ResultData.ranked` entries shape (with `skipped` unset, since a +/// range page has no rank base). +pub const DRIVE_ABCI_QUERY_VERSIONS_V3: DriveAbciQueryVersions = DriveAbciQueryVersions { + document_query_helpers: DriveAbciDocumentQueryHelperVersions { + compute_aggregate_mode_and_check_limit: 2, + }, + ..DRIVE_ABCI_QUERY_VERSIONS_V2 +}; diff --git a/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/mod.rs b/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/mod.rs index e7280615772..bdce22d77c9 100644 --- a/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/mod.rs +++ b/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/mod.rs @@ -41,6 +41,14 @@ pub struct DriveDocumentQueryMethodVersions { /// versions; the routing itself is unreachable before the ranked /// contract grammar activates. pub detect_ranked_mode: FeatureVersion, + /// Mode-detection routing table for boolean `HAVING` range queries + /// (`GROUP BY p HAVING LIMIT n`) served from an + /// indexed tree's axis secondary. Same versioning rationale and + /// same dormancy pattern as `detect_ranked_mode`: the slot exists + /// in every table, and the routing is unreachable before both the + /// ranked contract grammar and the v2 aggregate-routing helper + /// activate (protocol v14). + pub detect_having_mode: FeatureVersion, /// Lowering of a `DriveDocumentQuery` over a secondary index into a /// grovedb `PathQuery`. Versioned because the set of accepted query /// shapes is part of the consensus query contract: v0 rejects more diff --git a/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/v1.rs b/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/v1.rs index 004078759dd..7f84c0c17c7 100644 --- a/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/v1.rs +++ b/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/v1.rs @@ -18,6 +18,7 @@ pub const DRIVE_DOCUMENT_METHOD_VERSIONS_V1: DriveDocumentMethodVersions = detect_count_mode: 0, detect_sum_mode: 0, detect_ranked_mode: 0, + detect_having_mode: 0, non_primary_key_path_query: 0, where_clause_grouping: 0, }, diff --git a/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/v2.rs b/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/v2.rs index f1cfbe10f40..52d2e837189 100644 --- a/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/v2.rs +++ b/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/v2.rs @@ -20,6 +20,7 @@ pub const DRIVE_DOCUMENT_METHOD_VERSIONS_V2: DriveDocumentMethodVersions = detect_count_mode: 0, detect_sum_mode: 0, detect_ranked_mode: 0, + detect_having_mode: 0, non_primary_key_path_query: 0, where_clause_grouping: 0, }, diff --git a/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/v3.rs b/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/v3.rs index 31966e7f684..f3e1140f82f 100644 --- a/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/v3.rs +++ b/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/v3.rs @@ -30,6 +30,7 @@ pub const DRIVE_DOCUMENT_METHOD_VERSIONS_V3: DriveDocumentMethodVersions = detect_count_mode: 0, detect_sum_mode: 0, detect_ranked_mode: 0, + detect_having_mode: 0, non_primary_key_path_query: 0, where_clause_grouping: 0, }, diff --git a/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/v4.rs b/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/v4.rs index b57236bac37..2edcf0a1069 100644 --- a/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/v4.rs +++ b/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/v4.rs @@ -74,6 +74,7 @@ pub const DRIVE_DOCUMENT_METHOD_VERSIONS_V4: DriveDocumentMethodVersions = detect_count_mode: 0, detect_sum_mode: 0, detect_ranked_mode: 0, + detect_having_mode: 0, non_primary_key_path_query: 1, where_clause_grouping: 1, }, diff --git a/packages/rs-platform-version/src/version/drive_versions/drive_verify_method_versions/mod.rs b/packages/rs-platform-version/src/version/drive_versions/drive_verify_method_versions/mod.rs index 83e293a9c7e..84af8082e48 100644 --- a/packages/rs-platform-version/src/version/drive_versions/drive_verify_method_versions/mod.rs +++ b/packages/rs-platform-version/src/version/drive_versions/drive_verify_method_versions/mod.rs @@ -73,13 +73,15 @@ pub struct DriveVerifyDocumentSumMethodVersions { pub verify_point_lookup_count_and_sum_proof: FeatureVersion, } -/// Versions for the ranked-aggregate (`HAVING ... TOP/BOTTOM/MIN/MAX`) -/// prove-path verifier. The single method is implemented on -/// `DriveDocumentRankedQuery` and returns `(RootHash, Vec)`, -/// delegating to grovedb's `verify_indexed_axis_top_k`. +/// Versions for the indexed-axis prove-path verifiers: the ranked +/// (top-k) verifier and the boolean-`HAVING` range verifier. Both are +/// implemented on the respective drive query types and delegate to +/// grovedb's indexed-axis proof verification +/// (`verify_indexed_axis_top_k_paginated` / `verify_indexed_axis_query`). #[derive(Clone, Debug, Default)] pub struct DriveVerifyDocumentRankedMethodVersions { pub verify_ranked_top_k_proof: FeatureVersion, + pub verify_having_range_proof: FeatureVersion, } #[derive(Clone, Debug, Default)] diff --git a/packages/rs-platform-version/src/version/drive_versions/drive_verify_method_versions/v1.rs b/packages/rs-platform-version/src/version/drive_versions/drive_verify_method_versions/v1.rs index 30e0adb14f8..b9412b58615 100644 --- a/packages/rs-platform-version/src/version/drive_versions/drive_verify_method_versions/v1.rs +++ b/packages/rs-platform-version/src/version/drive_versions/drive_verify_method_versions/v1.rs @@ -41,6 +41,7 @@ pub const DRIVE_VERIFY_METHOD_VERSIONS_V1: DriveVerifyMethodVersions = DriveVeri }, document_ranked: DriveVerifyDocumentRankedMethodVersions { verify_ranked_top_k_proof: 0, + verify_having_range_proof: 0, }, identity: DriveVerifyIdentityMethodVersions { verify_full_identities_by_public_key_hashes: 0, diff --git a/packages/rs-platform-version/src/version/v14.rs b/packages/rs-platform-version/src/version/v14.rs index 21000350ca7..cfa1f2a0efc 100644 --- a/packages/rs-platform-version/src/version/v14.rs +++ b/packages/rs-platform-version/src/version/v14.rs @@ -16,7 +16,7 @@ use crate::version::dpp_versions::dpp_voting_versions::v2::VOTING_VERSION_V2; use crate::version::dpp_versions::DPPVersion; use crate::version::drive_abci_versions::drive_abci_checkpoint_parameters::v1::DRIVE_ABCI_CHECKPOINT_PARAMETERS_V1; use crate::version::drive_abci_versions::drive_abci_method_versions::v9::DRIVE_ABCI_METHOD_VERSIONS_V9; -use crate::version::drive_abci_versions::drive_abci_query_versions::v2::DRIVE_ABCI_QUERY_VERSIONS_V2; +use crate::version::drive_abci_versions::drive_abci_query_versions::v3::DRIVE_ABCI_QUERY_VERSIONS_V3; use crate::version::drive_abci_versions::drive_abci_structure_versions::v1::DRIVE_ABCI_STRUCTURE_VERSIONS_V1; use crate::version::drive_abci_versions::drive_abci_validation_versions::v10::DRIVE_ABCI_VALIDATION_VERSIONS_V10; use crate::version::drive_abci_versions::drive_abci_withdrawal_constants::v2::DRIVE_ABCI_WITHDRAWAL_CONSTANTS_V2; @@ -95,14 +95,17 @@ pub const PROTOCOL_VERSION_14: ProtocolVersion = 14; /// `verify_ranked_top_k_proof`. All are 0 today. The same table bumps the /// four index walkers to v2 and the document update walker to v1 for the /// shared-prefix fix. -/// * `DRIVE_ABCI_QUERY_VERSIONS_V2` bumps -/// `document_query_helpers.compute_aggregate_mode_and_check_limit` 0 → 1, -/// opening the ranked path on the v1 document-query handler: a grouped -/// aggregate whose single `order_by` names the selected aggregate -/// (`ORDER BY [ASC|DESC] LIMIT n [OFFSET m]`) routes to the ranked -/// executor. v13 and earlier keep the v1 table and therefore keep -/// rejecting that shape, so mixed-version networks agree across the -/// upgrade. +/// * `DRIVE_ABCI_QUERY_VERSIONS_V3` bumps +/// `document_query_helpers.compute_aggregate_mode_and_check_limit` 0 → 2, +/// opening two routes on the v1 document-query handler: the ranked path +/// (a grouped aggregate whose single `order_by` names the selected +/// aggregate — `ORDER BY [ASC|DESC] LIMIT n [OFFSET m]`) and the +/// boolean-`HAVING` range path (a grouped aggregate carrying exactly one +/// `having` clause on the selected aggregate — `GROUP BY p HAVING +/// LIMIT n`), the latter served as a value-bounded range +/// read of the covering ranked index's axis secondary. v13 and earlier +/// keep the v1 table and therefore keep rejecting both shapes, so +/// mixed-version networks agree across the upgrade. /// * `DRIVE_ABCI_VALIDATION_VERSIONS_V10` bumps /// `document_create_transition_structure_validation` 0 → 1, requiring a /// contested create transition's prefunded voting balance to name the @@ -126,7 +129,7 @@ pub const PLATFORM_V14: PlatformVersion = PlatformVersion { methods: DRIVE_ABCI_METHOD_VERSIONS_V9, validation_and_processing: DRIVE_ABCI_VALIDATION_VERSIONS_V10, // changed: contested-index cross-check + refersTo document reference validation withdrawal_constants: DRIVE_ABCI_WITHDRAWAL_CONSTANTS_V2, - query: DRIVE_ABCI_QUERY_VERSIONS_V2, // changed: ranked HAVING routing gate + query: DRIVE_ABCI_QUERY_VERSIONS_V3, // changed: ranked + boolean-HAVING routing gate checkpoints: DRIVE_ABCI_CHECKPOINT_PARAMETERS_V1, }, dpp: DPPVersion { @@ -158,16 +161,18 @@ mod tests { use super::*; use crate::version::v13::PLATFORM_V13; - /// The ranked-HAVING routing gate lives in v14's own query table, so - /// flipping it to feature version 1 touches only v14: a v13 node keeps - /// running the v0 helper, which rejects every non-empty HAVING, so a + /// The ranked / boolean-HAVING routing gate lives in v14's own query + /// table, so flipping it touches only v14: a v13 node keeps running + /// the v0 helper, which rejects every non-empty HAVING, so a /// mixed-version network agrees until the upgrade vote carries. /// - /// The flip is real as of the ranked routing landing — v14 selects the - /// v1 helper, which routes a single ranking-operand HAVING clause to - /// `dispatch_ranked_v1`. A change that made v13 non-zero here would be - /// consensus-breaking for already-deployed nodes, which is exactly what - /// the v13 half of this assertion guards. + /// v14 selects the v2 helper, which routes the ranked shape + /// (`ORDER BY LIMIT n`) to `dispatch_ranked_v1` and the + /// boolean-HAVING range shape (exactly one `having` clause on the + /// selected aggregate) to `dispatch_having_v1`. A change that made + /// v13 non-zero here would be consensus-breaking for + /// already-deployed nodes, which is exactly what the v13 half of + /// this assertion guards. #[test] fn ranked_having_routing_gate_is_v14_only() { assert_eq!( @@ -184,7 +189,7 @@ mod tests { .query .document_query_helpers .compute_aggregate_mode_and_check_limit, - 1 + 2 ); } @@ -266,6 +271,10 @@ mod tests { PLATFORM_V14.drive.methods.document.query.detect_ranked_mode, 0 ); + assert_eq!( + PLATFORM_V14.drive.methods.document.query.detect_having_mode, + 0 + ); assert_eq!( PLATFORM_V14 .drive @@ -275,6 +284,15 @@ mod tests { .verify_ranked_top_k_proof, 0 ); + assert_eq!( + PLATFORM_V14 + .drive + .methods + .verify + .document_ranked + .verify_having_range_proof, + 0 + ); let grove = &PLATFORM_V14.drive.grove_methods.batch; assert_eq!(grove.batch_insert_empty_provable_count_indexed_tree, 0); assert_eq!(grove.batch_insert_empty_provable_sum_indexed_tree, 0); diff --git a/packages/rs-sdk/src/mock/requests.rs b/packages/rs-sdk/src/mock/requests.rs index 7cb0696ac20..9015c19a583 100644 --- a/packages/rs-sdk/src/mock/requests.rs +++ b/packages/rs-sdk/src/mock/requests.rs @@ -789,3 +789,27 @@ impl MockResponse for drive_proof_verifier::DocumentRankedEntries { } } } + +impl MockResponse for drive_proof_verifier::DocumentHavingEntries { + /// Rides the ranked page encoding with a starting rank of `0`: a + /// having page is the same ordered `(group key, axis tag, value)` + /// list, just addressed by value bound instead of by rank, and it + /// has no rank base to preserve. + fn mock_serialize(&self, sdk: &MockDashPlatformSdk) -> Vec { + drive_proof_verifier::DocumentRankedEntries { + starting_rank: 0, + entries: self.entries.clone(), + } + .mock_serialize(sdk) + } + + fn mock_deserialize(sdk: &MockDashPlatformSdk, buf: &[u8]) -> Self + where + Self: Sized, + { + let page = drive_proof_verifier::DocumentRankedEntries::mock_deserialize(sdk, buf); + drive_proof_verifier::DocumentHavingEntries { + entries: page.entries, + } + } +} diff --git a/packages/rs-sdk/src/platform/documents/document_having_entries.rs b/packages/rs-sdk/src/platform/documents/document_having_entries.rs new file mode 100644 index 00000000000..d8349ca5ef5 --- /dev/null +++ b/packages/rs-sdk/src/platform/documents/document_having_entries.rs @@ -0,0 +1,323 @@ +//! `FromProof` + `Fetch` for [`DocumentHavingEntries`] — the +//! **having-range** (`GROUP BY … HAVING +//! LIMIT n`) view of the unified `getDocuments` endpoint. +//! +//! A having-range query answers "which groups' aggregate falls inside a +//! value bound?" — *hashtags with more than 100 posts* — in +//! `O(log n + k)`, with a proof whose Merk range boundaries also attest +//! **completeness**: a node cannot silently omit a matching group. It +//! reads the same pre-sorted per-axis *secondary* Merk the ranked +//! surface walks (grovedb PR #657), addressed by value bound instead of +//! by rank. +//! +//! Per-request resolution (which axis, which bounds the operator +//! translates to, which index covers them) lives in +//! [`super::having_proof_helpers`]; this module is the thin +//! `Fetch`-side wrapper. +//! +//! ## Request shape +//! +//! Exactly one aggregate `select`, exactly one `group_by` property, +//! exactly one `having` clause **bounding the selected aggregate** with +//! a contiguous-range operator (`=`, `>`, `>=`, `<`, `<=`, `BETWEEN*` — +//! `!=` and `IN` are rejected), and a `LIMIT`. `ORDER BY` is optional: +//! omitted means ascending by the aggregate; naming the selected +//! aggregate sets the direction. No `where`, no `offset`, no +//! `start_at`. +//! +//! ## Contract prerequisites +//! +//! Same as the ranked surface: the index must opt in with +//! `rankedCountable` / `rankedSummable` / `rankedAverageable` +//! (meta-schema v3, **protocol version 14+**), and ranked indexes are +//! single-property. Against a pre-v14 node the request is refused with +//! "HAVING clause is not yet implemented" — the intended activation +//! gate. +//! +//! ## Reading the result +//! +//! Entries come back in axis order in the walk direction; **do not +//! re-sort**. Fewer than `n` entries means fewer groups matched. +//! **Exactly `n` may mean the match set was cut at the limit.** +//! Tightening the bound past the last aggregate value seen continues +//! past *distinct* values only: a cut inside a tie (several groups +//! sharing the boundary aggregate) cannot be continued — the tied +//! groups past the limit stay unreachable until a composite-key cursor +//! exists — so size the limit above the widest expected tie. Averages +//! are fixed-point integers, exact on this (proved) path; see the +//! ranked module's notes, which apply verbatim. +//! +//! ## Example: hashtags with more than 100 posts +//! +//! `SELECT COUNT(*) GROUP BY hashtag HAVING $count > 100 ORDER BY $count DESC LIMIT 100` +//! +//! ```rust,no_run +//! use dash_sdk::{Sdk, platform::{DataContract, DocumentQuery, Fetch, Identifier}}; +//! use dash_sdk::drive::query::{ +//! HavingAggregate, HavingAggregateFunction, HavingClause, HavingOperator, +//! HavingRightOperand, SelectProjection, +//! }; +//! use dash_sdk::platform::documents::document_query::RankingDirection; +//! use dpp::platform_value::Value; +//! use drive_proof_verifier::DocumentHavingEntries; +//! use futures::executor::block_on; +//! +//! # const POSTS_CONTRACT_ID: [u8; 32] = [0; 32]; +//! let sdk = Sdk::new_mock(); +//! let contract = block_on(DataContract::fetch(&sdk, Identifier::new(POSTS_CONTRACT_ID))) +//! .expect("fetch contract") +//! .expect("contract exists"); +//! +//! let query = DocumentQuery::new(contract, "post") +//! .expect("document type exists") +//! .with_select(SelectProjection::count_star()) +//! .with_group_by("hashtag") +//! .with_having(vec![HavingClause { +//! aggregate: HavingAggregate { +//! function: HavingAggregateFunction::Count, +//! field: String::new(), +//! }, +//! operator: HavingOperator::GreaterThan, +//! right: HavingRightOperand::Value(Value::U64(100)), +//! }]) +//! .order_by_selected_aggregate(RankingDirection::Descending) +//! .with_limit(100); +//! +//! let matching = block_on(DocumentHavingEntries::fetch(&sdk, query)) +//! .expect("fetch succeeds") +//! .expect("a well-formed having query always answers"); +//! +//! for entry in &matching.entries { +//! let hashtag = String::from_utf8_lossy(&entry.key); +//! println!("#{hashtag}: {} posts", entry.value.as_f64()); +//! } +//! ``` + +use crate::platform::documents::document_query::DocumentQuery; +use crate::platform::documents::having_proof_helpers::verify_having_query; +use crate::platform::Fetch; +use dapi_grpc::platform::v0::{GetDocumentsResponse, Proof, ResponseMetadata}; +use dash_context_provider::ContextProvider; +use dpp::dashcore::Network; +use dpp::version::PlatformVersion; +use drive_proof_verifier::{DocumentHavingEntries, FromProof}; + +impl FromProof for DocumentHavingEntries { + type Request = DocumentQuery; + type Response = GetDocumentsResponse; + + fn maybe_from_proof_with_metadata<'a, I: Into, O: Into>( + request: I, + response: O, + _network: Network, + platform_version: &PlatformVersion, + provider: &'a dyn ContextProvider, + ) -> Result<(Option, ResponseMetadata, Proof), drive_proof_verifier::Error> + where + Self: 'a, + { + let request: Self::Request = request.into(); + let response: Self::Response = response.into(); + // Same single-pass design as the ranked impl: the grammar check + // is the first step of resolution, inside the helper. + let (entries, mtd, proof) = + verify_having_query(request, response, platform_version, provider)?; + Ok(( + entries.map(DocumentHavingEntries::from_verified), + mtd, + proof, + )) + } +} + +impl Fetch for DocumentHavingEntries { + type Query = DocumentQuery; + type Request = dapi_grpc::platform::v0::GetDocumentsRequest; +} + +#[cfg(test)] +mod tests { + //! Offline tests for the having client surface: the request→wire + //! encoding and the client-side grammar mirror. Proof verification + //! is exercised end-to-end in rs-drive's + //! `drive_document_having_query::tests` and rs-drive-abci's + //! `having_range_tests`, where a populated Drive exists. + + use super::*; + use crate::platform::documents::document_query::RankingDirection; + use crate::platform::documents::having_proof_helpers::assert_having_shape; + use dapi_grpc::platform::v0::get_documents_request::get_documents_request_v1::select as proto_select; + use dapi_grpc::platform::v0::get_documents_request::{ + having_aggregate, having_clause, GetDocumentsRequestV1, Version as RequestVersion, + }; + use dapi_grpc::platform::v0::GetDocumentsRequest; + use dpp::data_contract::DataContract; + use dpp::platform_value::Value; + use dpp::tests::fixtures::get_data_contract_fixture; + use dpp::version::TryFromPlatformVersioned; + use drive::query::{ + AxisRangeBounds, HavingAggregate, HavingAggregateFunction, HavingClause, HavingOperator, + HavingRightOperand, SelectProjection, + }; + use std::sync::Arc; + + fn platform_version() -> &'static PlatformVersion { + PlatformVersion::latest() + } + + fn contract() -> Arc { + Arc::new( + get_data_contract_fixture(None, 0, platform_version().protocol_version) + .data_contract_owned(), + ) + } + + fn count_over_100() -> HavingClause { + HavingClause { + aggregate: HavingAggregate { + function: HavingAggregateFunction::Count, + field: String::new(), + }, + operator: HavingOperator::GreaterThan, + right: HavingRightOperand::Value(Value::U64(100)), + } + } + + /// `SELECT COUNT(*) GROUP BY hashtag HAVING $count > 100 LIMIT 100`. + fn hashtags_over_100() -> DocumentQuery { + DocumentQuery::new(contract(), "niceDocument") + .expect("the fixture has this document type") + .with_select(SelectProjection::count_star()) + .with_group_by("hashtag") + .with_having(vec![count_over_100()]) + .with_limit(100) + } + + fn v1_of(query: DocumentQuery) -> GetDocumentsRequestV1 { + let request = GetDocumentsRequest::try_from_platform_versioned(query, platform_version()) + .expect("a having query encodes onto the V1 wire"); + match request.version.expect("the encoder always sets a version") { + RequestVersion::V1(v1) => v1, + RequestVersion::V0(_) => { + panic!("a having query must encode onto the V1 wire; V0 has no `having` field") + } + } + } + + /// The headline round-trip: the wire shape must be exactly what the + /// server's routing accepts — one select, one group_by, one having + /// clause, a limit, nothing else. + #[test] + fn having_query_encodes_the_expected_wire_shape() { + let v1 = v1_of(hashtags_over_100()); + + assert_eq!(v1.selects.len(), 1); + assert_eq!(v1.selects[0].function, proto_select::Function::Count as i32); + assert_eq!(v1.selects[0].field, ""); + assert_eq!(v1.group_by, vec!["hashtag".to_string()]); + + assert_eq!(v1.having.len(), 1, "exactly one having clause"); + let clause = &v1.having[0]; + let aggregate = clause.aggregate.as_ref().expect("aggregate is set"); + assert_eq!(aggregate.function, having_aggregate::Function::Count as i32); + assert_eq!(aggregate.field, ""); + assert_eq!(clause.operator, having_clause::Operator::GreaterThan as i32); + assert!(clause.right.is_some(), "the right operand rides the oneof"); + + assert_eq!(v1.limit, Some(100)); + assert!(v1.where_clauses.is_empty()); + assert!( + v1.order_by.is_empty(), + "order_by is optional and unset here" + ); + assert_eq!(v1.offset, None); + assert!(v1.start.is_none()); + assert!(v1.prove, "the Fetch path always requests a proof"); + } + + /// The client-side grammar must resolve the same bounds the server + /// (and therefore the prover) resolves — the bounds are rebuilt + /// into the proof's Merk query at verification time, so a client + /// that translated `> 100` differently could not verify an honest + /// proof. + #[test] + fn assert_having_shape_resolves_the_bounds() { + let mode = assert_having_shape(&hashtags_over_100(), platform_version()) + .expect("the headline query is well-formed"); + assert_eq!( + mode.bounds, + AxisRangeBounds::Count { + lo: 101, + hi: u64::MAX + } + ); + assert!(!mode.descending, "no order_by means ascending"); + assert_eq!(mode.limit, 100); + assert_eq!(mode.group_by_property, "hashtag"); + } + + /// An explicit descending ordering on the selected aggregate flips + /// the walk; biggest matching groups come first. + #[test] + fn ordering_by_the_aggregate_sets_the_direction() { + let query = hashtags_over_100().order_by_selected_aggregate(RankingDirection::Descending); + let mode = assert_having_shape(&query, platform_version()) + .expect("having + ORDER BY the aggregate is well-formed"); + assert!(mode.descending); + } + + /// Every knob the range walk cannot honour is rejected client side, + /// before a round trip — mirroring the server's rejections. + #[test] + fn assert_having_shape_rejects_what_the_range_cannot_honour() { + let base = hashtags_over_100(); + + // No having at all: a plain grouped aggregate. + let mut no_having = base.clone(); + no_having.having = Vec::new(); + assert!(assert_having_shape(&no_having, platform_version()).is_err()); + + // Two clauses: implicit AND is a future capability. + let two = base + .clone() + .with_having(vec![count_over_100(), count_over_100()]); + assert!(assert_having_shape(&two, platform_version()).is_err()); + + // A clause on a different aggregate than the select. + let cross = base.clone().with_having(vec![HavingClause { + aggregate: HavingAggregate { + function: HavingAggregateFunction::Sum, + field: "amount".to_string(), + }, + operator: HavingOperator::GreaterThan, + right: HavingRightOperand::Value(Value::I64(100)), + }]); + assert!(assert_having_shape(&cross, platform_version()).is_err()); + + // An offset: the range walk has no skip. + let with_offset = base.clone().with_offset(4); + assert!(assert_having_shape(&with_offset, platform_version()).is_err()); + + // Non-contiguous operators. + for operator in [HavingOperator::NotEqual, HavingOperator::In] { + let mut clause = count_over_100(); + clause.operator = operator; + let query = base.clone().with_having(vec![clause]); + assert!(assert_having_shape(&query, platform_version()).is_err()); + } + } + + /// The generic FromProof guard in drive-proof-verifier must not be + /// reachable from the SDK path: this impl (on `DocumentQuery`) is + /// the one `fetch` resolves, and it runs the real verification. + #[test] + fn limit_is_required_and_capped_client_side() { + for limit in [0u32, 101] { + let query = hashtags_over_100().with_limit(limit); + assert!( + assert_having_shape(&query, platform_version()).is_err(), + "LIMIT {limit} is outside 1..=100 and must be rejected, not clamped" + ); + } + } +} diff --git a/packages/rs-sdk/src/platform/documents/document_query.rs b/packages/rs-sdk/src/platform/documents/document_query.rs index ebf0a3a0dbf..679dcd9f2c9 100644 --- a/packages/rs-sdk/src/platform/documents/document_query.rs +++ b/packages/rs-sdk/src/platform/documents/document_query.rs @@ -89,16 +89,26 @@ pub struct DocumentQuery { /// [`drive::query::HavingOperator`] for the catalogs. Multiple /// entries combine with implicit `AND`. /// - /// **Every non-empty value is rejected by the server** with - /// `QuerySyntaxError::Unsupported("HAVING clause is not yet - /// implemented")`, at every protocol version. The typed builder - /// exists so callers can encode `HAVING` ahead of server support - /// landing without a wire-format change. + /// **Served from protocol version 14, for exactly one clause + /// bounding the selected aggregate** with a contiguous-range + /// operator (`=`, `>`, `>=`, `<`, `<=`, `BETWEEN*`) — the + /// having-range surface, fetched as + /// [`DocumentHavingEntries`](drive_proof_verifier::DocumentHavingEntries) + /// and served as a value-bounded range read of the covering ranked + /// index's axis secondary (the index must declare the matching + /// `rankedCountable` / `rankedSummable` / `rankedAverageable` + /// keyword). Everything else — multiple clauses (implicit AND), a + /// clause on an aggregate the select does not project, `!=` / `IN` + /// — is still rejected with `QuerySyntaxError::Unsupported`, as is + /// any non-empty value at protocol version 13 and earlier. /// /// **`having` does not express ranking.** "The n highest-scoring /// groups" is [`Self::order_by_selected_aggregate`] + /// [`Self::with_limit`] — SQL's own `ORDER BY DESC LIMIT n` - /// — which *is* served, from protocol version 14. + /// — which is also served from protocol version 14. The two + /// compose only in the one shape the having grammar allows: an + /// `ORDER BY` naming the selected aggregate sets the having + /// range's walk direction. #[cfg_attr(feature = "mocks", serde(default))] pub having: Vec, /// `order_by` clauses for the query. @@ -282,11 +292,17 @@ impl DocumentQuery { /// Set the `HAVING` clauses (replaces any prior value). /// - /// Non-empty values are rejected by the server with - /// `QuerySyntaxError::Unsupported("HAVING clause is not yet - /// implemented")`. The builder exists so SDK callers can - /// encode `HAVING` ahead of server support landing without - /// another version bump. + /// From protocol version 14, a grouped aggregate query carrying + /// **exactly one** clause that bounds the selected aggregate with + /// a contiguous-range operator (`=`, `>`, `>=`, `<`, `<=`, the + /// `BETWEEN` variants) is served as a value-bounded range read of + /// the covering ranked index's axis secondary — fetch the result + /// through `DocumentHavingEntries::fetch`, which verifies the + /// proof including its completeness. The server still rejects + /// multiple clauses, a clause on a different aggregate than the + /// select's, and the non-contiguous operators (`!=`, `IN`); + /// protocol version 13 and earlier reject every non-empty + /// `having`. /// /// This is **not** how you ask for a ranking — see /// [`Self::order_by_selected_aggregate`]. diff --git a/packages/rs-sdk/src/platform/documents/having_proof_helpers.rs b/packages/rs-sdk/src/platform/documents/having_proof_helpers.rs new file mode 100644 index 00000000000..2958f7ab0e7 --- /dev/null +++ b/packages/rs-sdk/src/platform/documents/having_proof_helpers.rs @@ -0,0 +1,162 @@ +//! Having-range proof dispatch used by [`DocumentHavingEntries`]. +//! +//! Having-side analog of [`super::ranked_proof_helpers`]: it turns a +//! caller-built [`DocumentQuery`] plus the node's response into a +//! verified entry list. The routing decisions — which axis, which +//! inclusive bounds the operator translates to, which direction, which +//! index covers them — are **not** re-derived here. They come from +//! rs-drive's own [`detect_having_mode`] and +//! [`find_ranked_index_for_axis`], the same two functions the server +//! calls, so client and server land on the same grove path and the same +//! bounds by construction rather than by two copies of a grammar +//! agreeing. The bounds matter doubly here: the verifier rebuilds the +//! proof's Merk query from them, so a divergence is a failed +//! verification, not a subtly different answer. +//! +//! [`DocumentHavingEntries`]: drive_proof_verifier::DocumentHavingEntries + +use crate::platform::documents::document_query::DocumentQuery; +use dapi_grpc::platform::v0::{GetDocumentsResponse, Proof, ResponseMetadata}; +use dapi_grpc::platform::VersionedGrpcResponse; +use dash_context_provider::ContextProvider; +use dpp::version::PlatformVersion; +use dpp::{ + data_contract::accessors::v0::DataContractV0Getters, + data_contract::document_type::accessors::DocumentTypeV0Getters, +}; +use drive::query::drive_document_having_query::mode_detection::detect_having_mode; +use drive::query::drive_document_ranked_query::index_picker::find_ranked_index_for_axis; +use drive::query::{ + DocumentHavingMode, DriveDocumentHavingQuery, RankedEntry, RankedPaginationInputs, +}; +use drive_proof_verifier::verify_having_range_proof; + +/// Validate that the caller-built [`DocumentQuery`] really describes a +/// having-range query, and resolve it into the `(bounds, descending, +/// limit, group property, aggregate field)` tuple the index picker and +/// the prover both work from. +/// +/// Same return-the-mode design as +/// [`assert_ranked_shape`](super::ranked_proof_helpers::assert_ranked_shape), +/// for the same reason: the grammar check *is* the first step of +/// resolution. The grammar lives in rs-drive ([`detect_having_mode`]) +/// and is versioned through +/// `platform_version.drive.methods.document.query.detect_having_mode`, +/// so the SDK cannot resolve a clause to different bounds than the +/// prover used. +pub(super) fn assert_having_shape( + request: &DocumentQuery, + platform_version: &PlatformVersion, +) -> Result { + // Same sentinel handling as the ranked helper: `limit == 0` is + // `DocumentQuery`'s "unset", reported to rs-drive as `None`. + let pagination = RankedPaginationInputs { + limit: (request.limit != 0).then_some(request.limit), + offset: request.offset, + has_start_at: request.start.is_some(), + }; + + detect_having_mode( + &request.select, + &request.group_by, + &request.having, + &request.order_by_clauses, + &request.where_clauses, + pagination, + platform_version, + ) + .map_err(|e| drive_proof_verifier::Error::RequestError { + error: format!( + "this DocumentQuery is not a well-formed having-range query: {e}. A having-range \ + query is `.with_select()`, `.with_group_by()`, \ + `.with_having()` and `.with_limit(n)`, optionally \ + `.order_by_selected_aggregate()`, with no where clauses, no offset \ + and no start_at." + ), + }) +} + +/// Verify a having-range proof and return the verified entries — the +/// matching groups **in axis order in the walk direction**. +/// +/// Single source of truth for the having proof path, mirroring +/// [`verify_ranked_query`](super::ranked_proof_helpers::verify_ranked_query) +/// step for step: re-run rs-drive's versioned request validation +/// (which resolves the bounds), resolve the covering index off the +/// contract, rebuild the query, verify. The root-hash binding to the +/// quorum-signed app hash happens inside [`verify_having_range_proof`] +/// and cannot be skipped through this helper. +pub(super) fn verify_having_query( + request: DocumentQuery, + response: GetDocumentsResponse, + platform_version: &PlatformVersion, + provider: &dyn ContextProvider, +) -> Result<(Option>, ResponseMetadata, Proof), drive_proof_verifier::Error> { + let document_type = request + .data_contract + .document_type_for_name(&request.document_type_name) + .map_err(|e| drive_proof_verifier::Error::RequestError { + error: format!( + "document type {} not found in contract: {}", + request.document_type_name, e + ), + })?; + let proof = response + .proof() + .or(Err(drive_proof_verifier::Error::NoProofInResult))?; + let mtd = response + .metadata() + .or(Err(drive_proof_verifier::Error::EmptyResponseMetadata))?; + + let mode = assert_having_shape(&request, platform_version)?; + let axis = mode.bounds.axis(); + + // Pick the index the prover picked — rs-drive's own picker, shared + // with the ranked surface because both read the same indexed tree. + let index = find_ranked_index_for_axis( + document_type.indexes(), + &mode.group_by_property, + axis, + &mode.aggregate_field, + ) + .ok_or_else(|| drive_proof_verifier::Error::RequestError { + error: format!( + "no index on document type `{}` can serve a `{:?}` having bound grouped on \ + `{}`: a having-range query needs a single-property index over `{}` declaring \ + `{}` (and, for SUM / AVG, `summable: \"{}\"`). Ranked indexes are opt-in \ + contract grammar (meta-schema v3, protocol version 14+).", + request.document_type_name, + axis, + mode.group_by_property, + mode.group_by_property, + axis.required_index_keyword(), + mode.aggregate_field, + ), + })?; + + let having_query = DriveDocumentHavingQuery { + document_type, + contract_id: request.data_contract.id().to_buffer(), + document_type_name: request.document_type_name.clone(), + index, + bounds: mode.bounds, + descending: mode.descending, + limit: mode.limit, + }; + + // Binds the reconstructed grovedb root hash to the quorum-signed + // app hash before returning — see the module docs. + let (root_hash, entries) = + verify_having_range_proof(&having_query, proof, mtd, platform_version, provider)?; + + tracing::trace!( + target: "dash_sdk::having_query", + root_hash = hex::encode(root_hash), + height = mtd.height, + entries = entries.len(), + "verified having range proof" + ); + + Ok((Some(entries), mtd.clone(), proof.clone())) +} diff --git a/packages/rs-sdk/src/platform/documents/mod.rs b/packages/rs-sdk/src/platform/documents/mod.rs index dbb6c5ae5ba..5f7531c7d51 100644 --- a/packages/rs-sdk/src/platform/documents/mod.rs +++ b/packages/rs-sdk/src/platform/documents/mod.rs @@ -4,6 +4,12 @@ pub(super) mod count_proof_helpers; /// `(count, sum)`; client divides. pub mod document_average; pub mod document_count; +/// `Fetch` impl for the having-range (`GROUP BY … HAVING +/// LIMIT n`) result — one entry per matching group, in +/// axis order, with proof-attested completeness. Requires an index +/// declaring `rankedCountable` / `rankedSummable` / `rankedAverageable` +/// (protocol version 14+). +pub mod document_having_entries; pub mod document_history_query; pub mod document_query; /// `Fetch` impl for the ranked (`GROUP BY … ORDER BY LIMIT n @@ -22,6 +28,7 @@ pub mod document_split_sums; /// `Fetch` impl for the sum-side aggregate result. Mirrors /// `document_count`. Lights up alongside grovedb PR 670. pub mod document_sum; +pub(super) mod having_proof_helpers; pub(super) mod ranked_proof_helpers; pub(super) mod sum_proof_helpers; pub mod transitions;