Optimize MongoDB If-None-Exist logic

[TheCaffinatedDeveloper]

Summary

MongoDB transaction Bundle handling currently appears to implement If-None-Exist conditional creates by loading all active resources of the requested resource type for the tenant and evaluating the conditional search parameters in application memory.

This bypasses the existing search_index collection and its SearchParameter-specific indexes.

For tenants with a large number of resources of a given type, this makes each conditional create potentially O(N) in the number of resources of that type and could become particularly expensive when If-None-Exist is used frequently during high-volume transaction Bundle ingestion.

Current behavior

find_matching_resources_in_bundle_transaction queries resources using only:

.find(doc! {
    "tenant_id": tenant_id,
    "resource_type": resource_type,
    "is_deleted": false,
})
.session(&mut *session)

It then collects the returned documents:

let docs = collect_session_documents(cursor, session).await?;

and evaluates the parsed conditional search parameters against those resources in application code.

For a request such as:

POST Patient
If-None-Exist: identifier=https://example.com/mrn|12345

a tenant with 1 million active Patient resources could therefore require HFS to retrieve and inspect a very large portion of those Patient documents simply to determine whether the identifier already exists.

Existing search infrastructure

MongoBackend already maintains the search_index collection, including indexes designed for the corresponding FHIR search types.

For example, token searches can use an index shaped like:

{
    tenant_id: 1,
    resource_type: 1,
    param_name: 1,
    value_token_system: 1,
    value_token_code: 1
}

The search-index writes/deletes are also already performed using the same Mongo ClientSession during transaction Bundle processing.

It therefore seems possible for conditional matching to query search_index within the existing transaction rather than enumerating the underlying resources.

For the identifier example above, the lookup could conceptually target:

{
    tenant_id: tenant_id,
    resource_type: "Patient",
    param_name: "identifier",
    value_token_system: "https://example.com/mrn",
    value_token_code: "12345"
}

and then resolve the resulting resource_id values.

Impact

This is potentially significant for workloads that:

• contain hundreds of thousands or millions of resources of the same type per tenant;
• make heavy use of If-None-Exist;
• ingest transaction Bundles concurrently; or
• have high concurrent read/write activity.

Besides the scan itself, retrieving and filtering the resources in HFS can increase Mongo reads, network traffic, application memory/CPU usage, and transaction duration.

Suggested improvement

Could find_matching_resources_in_bundle_transaction reuse the normal SearchParameter/search-index query machinery while executing those queries through the active ClientSession?

Ideally, conditional matching within a transaction would preserve the existing transaction semantics while using the same indexed access paths as normal FHIR searches.

If there is a reason the regular search_index cannot safely be used for conditional matching within the transaction, it would also be useful to understand that constraint.