From c27686eeb5a6c91cadfa6045f4049a663411a71f Mon Sep 17 00:00:00 2001 From: Karl Rankla Date: Tue, 25 Aug 2026 20:51:46 +0300 Subject: [PATCH 1/9] docs(integration-toolkit): document inbound direct mode New Direct Mode page under Inbound Integration: sending entity-shaped payloads that skip the mapping engine while keeping deduplication, ordering, create-vs-update resolution, relations, meter readings, and monitoring. Covers enabling the mode (open vs entity allowlist), the versioned payload contract, relations and relation references, the simulateDirect dry run, the DIRECT_* monitoring codes, what stays unsupported in version 1, and operational notes. Cross-linked from the Mapping page and the use case configuration guide; sidebar renumbered to slot the page right after Mapping. --- .../integration-toolkit/configuration.md | 1 + .../inbound/direct-mode.md | 493 ++++++++++++++++++ .../integration-toolkit/inbound/examples.md | 2 +- .../integration-toolkit/inbound/mapping.md | 4 + .../inbound/meter-readings.md | 2 +- .../integration-toolkit/inbound/pricing.md | 2 +- .../integration-toolkit/inbound/relations.md | 2 +- .../inbound/unique-identifiers.md | 2 +- 8 files changed, 503 insertions(+), 5 deletions(-) create mode 100644 docs/integrations/integration-toolkit/inbound/direct-mode.md diff --git a/docs/integrations/integration-toolkit/configuration.md b/docs/integrations/integration-toolkit/configuration.md index 6927f961..2cd44fa5 100644 --- a/docs/integrations/integration-toolkit/configuration.md +++ b/docs/integrations/integration-toolkit/configuration.md @@ -122,6 +122,7 @@ curl -X POST 'https://erp-integration.sls.epilot.io/v1/integrations/{integration | `configuration` | object | Yes | Type-specific configuration (see sections below) | :::info +- `inbound` — Mapped by default; set `"direct": true` in the configuration to skip mapping and send entity-shaped payloads directly. See [Direct Mode](./inbound/direct-mode.md). - `file_proxy` — On-demand file serving from external systems or outbound delivery of epilot files. See the [File Proxy](./file-proxy.md) and [Outbound File Delivery](./outbound-file-delivery.md) guides. - `managed_call` — Synchronous external API calls with JSONata mapping. See [Managed Call Use Cases](#managed-call-use-cases). - `secure_proxy` — Route requests through epilot's secure proxy for static IP or VPN access. See [Secure Proxy Use Cases](#secure-proxy-use-cases). diff --git a/docs/integrations/integration-toolkit/inbound/direct-mode.md b/docs/integrations/integration-toolkit/inbound/direct-mode.md new file mode 100644 index 00000000..a78d8438 --- /dev/null +++ b/docs/integrations/integration-toolkit/inbound/direct-mode.md @@ -0,0 +1,493 @@ +--- +sidebar_position: 3 +title: Direct Mode +description: Send entity-shaped payloads directly, skipping the mapping engine +--- + +# Direct Mode + +Direct mode lets your integration send payloads that are **already in entity-operation format**, skipping the mapping engine entirely. It is built for middleware that already produces entity-shaped data — for example an integration layer on your side that knows your epilot schemas and does its own transformation. + +Direct mode removes only the transformation step. Everything else in the inbound pipeline behaves exactly as in mapped mode: + +- Ingest deduplication (`deduplication_id`) +- Buffering, per-entity ordering, and fair processing +- Create-vs-update resolution against existing entities +- Relation and relation-reference resolution, including automatic stub creation +- Meter-reading matching +- Sync activity attachment and echo prevention +- Monitoring events + +What you give up is the mapping engine itself: field mappings, JSONata and JSONPath expressions, constants, and the mapping-only field types (see [Unsupported in Version 1](#unsupported-in-version-1)). If your source system emits raw ERP payloads that need transformation, use [Mapping](./mapping.md) instead. A single integration can freely mix direct and mapped use cases. + +## How It Works + +``` +Middleware sends POST /v3/erp/updates/events (payload = direct payload) + | +epilot resolves the use case configuration (direct: true) + | +The payload is validated — schema, version, entity allowlist + | (invalid events are rejected per-event, never processed) + | +The event is queued — deduplication, ordering, fairness unchanged + | +The operations are translated to internal entity / meter-reading updates + | (no mapping engine involved) + | +Create-vs-update, relations, and meter readings resolve as in mapped mode + | +Monitoring events are emitted (ENTITY_CREATED, ENTITY_UPDATED, ...) +``` + +## Enabling Direct Mode + +Direct mode is enabled per **inbound use case** by setting `direct: true` on the use case's `configuration`. All events routed to that use case are then interpreted as direct payloads — a use case is either fully direct or fully mapped, never both. + +The minimal configuration: + +```json title="Open mode — any entity may be written" +{ + "direct": true +} +``` + +With an optional entity allowlist: + +```json title="Allowlist mode" +{ + "direct": true, + "entities": [ + { "entity_schema": "contract", "unique_ids": ["contract_number"] }, + { "entity_schema": "contact", "unique_ids": ["customer_number"] } + ] +} +``` + +Notes: + +- Allowlist entries need no `fields` array — there is nothing to map. +- Declared `unique_ids` are checked against your real entity schemas when the use case is saved, so typos surface at design time. +- Direct mode requires the **v3 events endpoint** (`POST /v3/erp/updates/events` with `integration_id`). The deprecated v1/v2 endpoints do not support it. + +### Open Mode vs Entity Allowlist + +If `entities` is **absent or empty**, the use case runs in **open mode**: any `entity_slug` is accepted. + +If `entities` is **non-empty**, it acts as an allowlist for entity operations: + +1. The operation's `entity_slug` must equal the `entity_schema` of some allowlist entry, otherwise the operation is rejected with `DIRECT_ENTITY_NOT_ALLOWED`. +2. The operation's `unique_ids` keys must be **exactly the declared set** (order-insensitive), **or** exactly `["_id"]` — referencing an entity directly by its epilot ID is always allowed. Any other key set is rejected with `DIRECT_ENTITY_NOT_ALLOWED`, and the error message names the expected keys. + +Meter-reading operations are **not** allowlist-gated in version 1 — the allowlist applies to entity operations only. + +### Configuration Propagation + +Use case configurations are cached in the pipeline for up to **5 minutes**. Plan cutovers accordingly: do not start sending direct payloads to a freshly flipped use case until the window has passed (or accept that a few events may be rejected and need to be resent). + +## Payload Contract (Version "1") + +The direct payload travels inside the existing `ErpEventV3.payload` — either as a JSON object or as a serialized JSON string (with `format: "json"`). XML is rejected for direct use cases. + +The schema is snake_case throughout. All object schemas are **strict**: unknown keys are rejected with a clear error naming the offending path. The single exception is `attributes`, which is a free-form record — attribute values pass through verbatim, except relation envelopes, which are validated and translated (see [Relations](#relations)). + +### Versioning Policy + +The `version` field pins the payload schema: + +- Changes within a version are **additive-only** — existing payloads never break within version `"1"`. +- A future breaking revision arrives as version `"2"`, with both versions accepted in parallel. +- An unknown version is rejected with `DIRECT_VERSION_UNSUPPORTED`. + +### Envelope + +```json +{ + "version": "1", + "operations": [ + { + "entity_slug": "contact", + "unique_ids": { "customer_number": "42" }, + "attributes": { + "first_name": "Erika", + "last_name": "Mustermann" + } + } + ] +} +``` + +| Field | Required | Description | +|-------|----------|-------------| +| `version` | **Yes** | Payload schema version. Currently only `"1"` (string). Unknown values are rejected with `DIRECT_VERSION_UNSUPPORTED`. | +| `operations` | **Yes** | 1 to 100 operations, applied in order. Each is an [entity operation](#entity-operations) or a [meter reading operation](#meter-reading-operations). | + +### Entity Operations + +An operation with `type` omitted or set to `"entity"`: + +```json +{ + "type": "entity", + "entity_slug": "contract", + "unique_ids": { "contract_number": "C-123" }, + "mode": "upsert", + "attributes": { + "status": "active", + "contract_start_date": "2026-09-01" + } +} +``` + +| Field | Required | Description | +|-------|----------|-------------| +| `type` | No | `"entity"` (default when omitted). | +| `entity_slug` | **Yes** | Target epilot entity schema (e.g. `"contact"`, `"contract"`). Non-empty string. | +| `unique_ids` | **Yes** | Object with at least one key. Keys are attribute names used to find the existing entity; values are strings or numbers (numbers are coerced to strings). Values that are empty after trimming are rejected. Use the single key `"_id"` to reference an entity directly by its epilot ID. See [Unique Identifiers](./unique-identifiers.md) for lookup behavior. | +| `unique_id_types` | No | Per-field type hint, `"email"` or `"phone"` — overrides the server-side schema derivation. See [Email and Phone Unique Identifiers](#email-and-phone-unique-identifiers). | +| `mode` | No | `"upsert"` (default), `"delete"`, or `"purge"`. | +| `attributes` | **Yes** for `upsert` | The attributes to write, verbatim. Optional for `delete` / `purge`. Relation envelopes inside `attributes` are validated and translated (see [Relations](#relations)); everything else passes through untouched. | + +Unknown keys on the **operation object** are rejected (strict schema), while unknown keys **inside `attributes`** are free-form and pass through. + +Delete example — `attributes` may be omitted: + +```json +{ + "entity_slug": "contract", + "unique_ids": { "contract_number": "C-123" }, + "mode": "delete" +} +``` + +### Meter Reading Operations + +An operation with `type: "meter_reading"`: + +```json +{ + "type": "meter_reading", + "meter": { "unique_ids": { "meter_number": "M-42" } }, + "counter": { "unique_ids": { "obis_number": "1-0:1.8.0" } }, + "mode": "upsert", + "reading_matching": "strict-date", + "attributes": { + "external_id": "R-9", + "timestamp": "2026-08-24T06:00:00Z", + "source": "ERP", + "value": 12345.6, + "direction": "feed-in" + } +} +``` + +| Field | Required | Description | +|-------|----------|-------------| +| `type` | **Yes** | Must be `"meter_reading"` for this variant. | +| `meter` | **Yes** | `{ "unique_ids": { ... } }` with at least one key — identifies the meter. | +| `counter` | No | `{ "unique_ids": { ... } }` with at least one key when present — identifies the meter counter. | +| `mode` | No | `"upsert"` (default) or `"delete"`. | +| `reading_matching` | No | `"external_id"` or `"strict-date"`. See [Reading Matching Strategies](./meter-readings.md#reading-matching-strategies) for semantics. | +| `attributes` | **Yes** | The reading itself; see the table below. | + +Reading attributes: + +| Attribute | Required | Description | +|-----------|----------|-------------| +| `external_id` | **Yes** | String or number (numbers are coerced to strings). | +| `timestamp` | **Yes** | ISO-8601 date or datetime. | +| `source` | **Yes** | One of `ECP`, `ERP`, `360`, `journey-submission`. | +| `value` | **Yes** | Number, or a numeric string (coerced to a number). Non-numeric values are rejected. | +| `direction` | No | For example `"feed-in"`. | +| other keys | No | Extras such as `reason`, `read_by`, `status`, `metadata` pass through verbatim. | + +:::caution +`reading_matching: "strict-date"` **requires** `counter`. Date-based matching searches for existing readings on the counter — without a counter it can never find a match, so the combination is rejected at validation time. +::: + +## Relations + +Relations use the same envelope concepts as mapped mode, written directly as attribute values. Resolution, tags, and automatic stub creation behave exactly as described in [Relations](./relations.md). + +### Relation Operations + +On any attribute inside `attributes`, a relation value is either a bare array (shorthand for `_set`, i.e. replace) or an object with **exactly one** operation key: + +```json +"contacts": { "$relation": [ { "schema": "contact", "unique_ids": { "customer_number": "42" } } ] } +``` + +```json +"contacts": { "$relation": { "_append": [ { "schema": "contact", "unique_ids": { "customer_number": "43" } } ] } } +``` + +| Operation | Behavior | +|-----------|----------| +| `_set` (or bare array) | Replace the relation list. | +| `_append` | Merge into the existing list, deduplicated by entity ID. | +| `_append_all` | Merge into the existing list without deduplication. | + +**Only** `_set`, `_append`, and `_append_all` are accepted — any other key is rejected at validation time, so a typo like `_apend` fails fast with an actionable error instead of producing an unresolved relation downstream. + +### Relation Items + +Each item in the operation array is one of two forms. + +**Already-resolved reference** — you know the epilot entity ID: + +```json +{ "entity_id": "0195c3d2-7f4a-71b8-9e02-4c1a5d6e8f90", "tags": ["primary"] } +``` + +**Lookup by unique identifiers** — the pipeline resolves (or stub-creates) the target: + +```json +{ + "schema": "contact", + "unique_ids": { "customer_number": "42" }, + "tags": ["primary"] +} +``` + +| Field | Required | Description | +|-------|----------|-------------| +| `entity_id` | resolved form | The epilot entity ID of the target. | +| `schema` | lookup form | Entity schema of the target. | +| `unique_ids` | lookup form | At least one key; same value rules as top-level `unique_ids`. | +| `unique_id_types` | No | Per-field `"email"` / `"phone"` override, same as top level. | +| `tags` | No | Relation tags (labels), e.g. `["primary"]`. | + +Using `"_id"` as the **sole** key of `unique_ids` — at the top level of an entity operation or inside a relation lookup item — references the entity directly by its epilot ID, skipping the search. The `["_id"]` key set is always accepted by the [entity allowlist](#open-mode-vs-entity-allowlist). + +### Email and Phone Unique Identifiers + +Email and phone attributes are repeatable field types in epilot and need type-aware matching (see [Special Identifier Types](./unique-identifiers.md#special-identifier-types)). In direct mode this is handled **server-side**: the pipeline inspects the target entity schema and derives the field type for each unique identifier whose schema attribute has type `email` or `phone`. You normally do not need to do anything: + +```json +{ + "entity_slug": "contact", + "unique_ids": { "email": "erika@example.com" }, + "attributes": { "first_name": "Erika" } +} +``` + +To override or supplement the derivation, set `unique_id_types` explicitly — explicit values always win: + +```json +{ + "entity_slug": "contact", + "unique_ids": { "email": "erika@example.com" }, + "unique_id_types": { "email": "email" }, + "attributes": { "first_name": "Erika" } +} +``` + +Derivation applies to the top-level `unique_ids` **and** to relation / relation-reference lookup items (per item `schema`). If a schema cannot be resolved, derivation is skipped for that schema with a warning — explicit `unique_id_types` still apply, so set them explicitly for custom schemas you rely on. + +### Relation References + +A relation reference points at an **item inside a repeatable attribute** of another entity — for example one address out of a contact's `address` list. Same concept as [Relation References in mapped mode](./relations.md#relation-references), expressed directly: + +```json +"billing_address": { + "$relation_ref": { + "_append": [ + { + "schema": "contact", + "unique_ids": { "customer_number": "42" }, + "path": "address", + "value": { "street": "Main Street", "city": "Berlin" } + } + ] + } +} +``` + +| Field | Required | Description | +|-------|----------|-------------| +| `schema` | **Yes** | Entity schema of the target entity. | +| `unique_ids` | **Yes** | Identifies the target entity; same rules as relation lookup items. | +| `unique_id_types` | No | Per-field `"email"` / `"phone"` override. | +| `path` | **Yes** | Attribute on the **target** entity holding the repeatable array (e.g. `"address"`). | +| `value` | **Yes** | The item to match (or create) at that path. | + +The operation envelope follows the same rules as `$relation`: a bare array is shorthand for `_set`, and only `_set` / `_append` / `_append_all` are accepted. + +## Sending Direct Events + +Direct events use the **same endpoint** as mapped events: `POST /v3/erp/updates/events`. The only difference is the payload content — the routing (`use_case_slug` or `event_name`), timestamps, deduplication, and ordering fields are unchanged. + +```json +{ + "integration_id": "123e4567-e89b-12d3-a456-426614174000", + "events": [ + { + "use_case_slug": "customer-sync", + "timestamp": "2026-08-24T06:00:00Z", + "format": "json", + "deduplication_id": "customer-42-2026-08-24T060000-001", + "payload": { + "version": "1", + "operations": [ + { + "entity_slug": "contact", + "unique_ids": { "customer_number": "42" }, + "attributes": { + "first_name": "Erika", + "last_name": "Mustermann" + } + }, + { + "entity_slug": "contract", + "unique_ids": { "contract_number": "C-123" }, + "attributes": { + "status": "active", + "contacts": { + "$relation": [ + { "schema": "contact", "unique_ids": { "customer_number": "42" }, "tags": ["primary"] } + ] + } + } + } + ] + } + } + ] +} +``` + +The `payload` may also be a serialized JSON string (per the `ErpEventV3` contract); it is parsed on arrival. `format: "xml"` is rejected for direct use cases. + +### Deduplication + +`deduplication_id` works exactly as in mapped mode — see [Deduplication](../configuration.md#deduplication). + +One caveat matters more in direct mode: the pipeline additionally applies **content-based deduplication** — two **byte-identical** events within a 5-minute window collapse silently into one. Direct payloads are far more likely to be byte-identical than raw ERP payloads (no incidental timestamps or sequence fields from the source system). If your middleware can legitimately send the same operation twice in quick succession and both must be processed, make each event distinguishable — set a unique `deduplication_id` per logical event (for example, include a source sequence number or timestamp). + +### Ordering + +Ordering is derived exactly as in mapped mode: + +- The request-level and event-level `group_id` control cross-event parallelism (omit for strict per-integration ordering). +- Within the pipeline, per-entity ordering is derived from `entity_slug` + `unique_ids` — two operations targeting the same entity are always processed in order, regardless of which events carried them. + +### Validation and Failure Behavior + +Direct payloads are validated **fully on arrival** — schema, version, and allowlist — using the same code path the pipeline uses, so there is no drift between what the API accepts and what the pipeline processes. + +- A malformed request envelope (invalid `ErpUpdatesEventsV3Request`) is rejected with HTTP `400`. +- An event whose direct payload fails validation gets a per-event `status: "error"` in the response, and the overall request returns HTTP `422`. Rejected events are **never processed** — fix and resend. +- Error messages are actionable: they name the operation index and the field path, e.g. `operations[3].unique_ids: at least one unique identifier is required`. +- Each rejection also emits a monitoring event with the matching `DIRECT_*` code (see [Monitoring](#monitoring)). + +An event accepted at ingest can still fail inside the pipeline in one scenario: the use case configuration changed between acceptance and processing (within the [5-minute cache window](#configuration-propagation)). Such failures do not retry — the event is dropped and reported via a monitoring event with the matching `DIRECT_*` code. + +## Dry Run: simulateDirect + +`POST /v1/erp/updates/direct_simulation` (operation ID `simulateDirect`) validates a direct payload against a configuration **without persisting anything** — the direct-mode counterpart of `simulateMappingV2`. Use it while developing your middleware, and in CI against your fixture payloads. + +Request: + +```json +{ + "event_configuration": { + "direct": true, + "entities": [ + { "entity_schema": "contract", "unique_ids": ["contract_number"] } + ] + }, + "payload": { + "version": "1", + "operations": [ + { + "entity_slug": "contract", + "unique_ids": { "contract_number": "C-123" }, + "attributes": { "status": "active" } + }, + { + "entity_slug": "order", + "unique_ids": { "order_number": "O-9" }, + "attributes": { "status": "open" } + } + ] + } +} +``` + +Response — **all** errors are collected across operations (the simulation does not stop at the first failure, unlike live ingest where the event is all-or-nothing): + +```json +{ + "valid": false, + "errors": [ + { + "code": "DIRECT_ENTITY_NOT_ALLOWED", + "message": "operations[1]: entity_slug \"order\" is not allowed by the use case's entity allowlist", + "operation_index": 1 + } + ] +} +``` + +When the payload is valid, the response includes the translated internal update previews — exactly what the pipeline would process — plus schema warnings for unique identifiers that do not exist in the target schema (same design-time check as mapping simulation): + +```json +{ + "valid": true, + "errors": [], + "warnings": [ + { + "entity_schema": "contract", + "field": "erp_contract_key", + "message": "Unique identifier \"erp_contract_key\" not found in schema \"contract\"" + } + ], + "entity_updates": [ + { + "entity_slug": "contract", + "unique_identifiers": { "contract_number": "C-123" }, + "mode": "upsert", + "attributes": { "status": "active" } + } + ], + "meter_reading_updates": [] +} +``` + +Schema warnings do not fail the simulation — an unknown unique identifier is a warning because lookups on it will simply never match, causing every event to create a new entity. That is almost always a configuration mistake worth fixing before go-live. + +## Monitoring + +Direct mode adds three monitoring codes. All three are **error**-level: + +| Code | Level | Category | Meaning | +|------|-------|----------|---------| +| `DIRECT_PAYLOAD_INVALID` | error | validation | Direct payload failed schema validation, JSON parsing, or used the XML format. | +| `DIRECT_VERSION_UNSUPPORTED` | error | validation | Payload `version` is not in the supported set. | +| `DIRECT_ENTITY_NOT_ALLOWED` | error | configuration | `entity_slug` or `unique_ids` keys not permitted by the use case's entity allowlist. | + +Success paths reuse the existing codes — direct operations are indistinguishable from mapped ones once translated: `ENTITY_CREATED`, `ENTITY_UPDATED`, `ENTITY_DELETED`, `ENTITY_NO_OP`, `METER_READING_UPSERTED`, `METER_READING_DELETED`. + +## Unsupported in Version 1 + +The following are intentionally **out of scope** for direct mode version 1. In each case, mapped mode remains fully available — a single integration can mix direct and mapped use cases freely. + +| Not supported | Why | What to use instead | +|---------------|-----|---------------------| +| Prune-scope operations (`upsert-prune-scope-*`) | Destructive bulk semantics need their own design before being exposed on a raw wire format. | Mapped mode [Operation Modes](./mapping.md#operation-modes). | +| File proxy URL construction | The proxy URL embeds server-side context the integrator does not have. | Mapped mode [File Proxy URL Mapping](./mapping.md#file-proxy-url-mapping). | +| `portal_ref` | Resolved from server-side portal configuration. | Mapped mode [Portal Reference Mapping](./mapping.md#portal-ref-mapping). | +| `env_var_ref` | Environment variables and secrets are resolved server-side and must not round-trip through the integrator. | Mapped mode [Environment Variable Reference Mapping](./mapping.md#env-var-ref-mapping). | +| Pricing | Couples to the pricing engine's server-side product and price resolution. | Mapped mode [Pricing](./pricing.md). | +| XML payloads | The direct contract is JSON-only by design. | Send JSON; for XML-emitting sources use mapped mode. | +| CSV imports against direct use cases | CSV imports emit mapping-shaped events; routed to a direct use case they fail with `DIRECT_PAYLOAD_INVALID`. | Route CSV imports to mapped use cases. | +| v1/v2 events endpoints | The legacy configuration paths do not carry the `direct` flag. | `POST /v3/erp/updates/events` with `integration_id`. | +| Meter-reading allowlist gating | The entity allowlist covers entity operations only in version 1. | Open by design; gate entity operations if needed. | + +## Operational Notes + +- **Size budget.** Each event is processed as a single message with a 256 KiB limit (1 MiB for meter-reading batches). A payload holds at most **100 operations**; split larger batches across multiple events. The operation limit is additive to raise in a future revision if needed. +- **No entity-attribute validation — by design.** Attributes are written verbatim; attributes not defined in the schema are stored but not indexed. This is intentional and common integration practice (mapped mode behaves the same way). The design-time guards are [simulateDirect](#dry-run-simulatedirect) and the unique-identifier schema warnings. +- **Configuration propagation.** The `direct` flag takes up to 5 minutes to reach the pipeline after a configuration change. +- **Content-based deduplication.** Byte-identical events within 5 minutes collapse silently — set `deduplication_id` deliberately (see [Deduplication](#deduplication)). +- **Sync activities.** A sync activity is attached to every write automatically. Direct writes do **not** echo back out through outbound delivery — the same echo prevention as mapped mode. diff --git a/docs/integrations/integration-toolkit/inbound/examples.md b/docs/integrations/integration-toolkit/inbound/examples.md index 96b9bcd6..ee679c91 100644 --- a/docs/integrations/integration-toolkit/inbound/examples.md +++ b/docs/integrations/integration-toolkit/inbound/examples.md @@ -1,5 +1,5 @@ --- -sidebar_position: 7 +sidebar_position: 8 title: Examples description: Complete integration examples for common use cases --- diff --git a/docs/integrations/integration-toolkit/inbound/mapping.md b/docs/integrations/integration-toolkit/inbound/mapping.md index bc1942fc..e1104f83 100644 --- a/docs/integrations/integration-toolkit/inbound/mapping.md +++ b/docs/integrations/integration-toolkit/inbound/mapping.md @@ -8,6 +8,10 @@ description: Configure how ERP data transforms into epilot entities Mapping defines how ERP data transforms into epilot entities. This page covers the configuration structure and available options. +:::tip +If your middleware already produces entity-shaped payloads, you can skip mapping entirely — see [Direct Mode](./direct-mode.md). +::: + ## Mapping Configuration A mapping configuration consists of one or more entity definitions: diff --git a/docs/integrations/integration-toolkit/inbound/meter-readings.md b/docs/integrations/integration-toolkit/inbound/meter-readings.md index d24185ff..97e691d7 100644 --- a/docs/integrations/integration-toolkit/inbound/meter-readings.md +++ b/docs/integrations/integration-toolkit/inbound/meter-readings.md @@ -1,5 +1,5 @@ --- -sidebar_position: 6 +sidebar_position: 7 title: Meter Readings description: Synchronize meter reading data from ERP systems --- diff --git a/docs/integrations/integration-toolkit/inbound/pricing.md b/docs/integrations/integration-toolkit/inbound/pricing.md index 57ccdcad..e18b227e 100644 --- a/docs/integrations/integration-toolkit/inbound/pricing.md +++ b/docs/integrations/integration-toolkit/inbound/pricing.md @@ -1,5 +1,5 @@ --- -sidebar_position: 5 +sidebar_position: 6 title: Pricing description: Map ERP line items and calculate prices during inbound synchronization --- diff --git a/docs/integrations/integration-toolkit/inbound/relations.md b/docs/integrations/integration-toolkit/inbound/relations.md index 31b810ca..470b3e30 100644 --- a/docs/integrations/integration-toolkit/inbound/relations.md +++ b/docs/integrations/integration-toolkit/inbound/relations.md @@ -1,5 +1,5 @@ --- -sidebar_position: 4 +sidebar_position: 5 title: Relations description: Link entities together during synchronization --- diff --git a/docs/integrations/integration-toolkit/inbound/unique-identifiers.md b/docs/integrations/integration-toolkit/inbound/unique-identifiers.md index 4ee37fac..1f0018ce 100644 --- a/docs/integrations/integration-toolkit/inbound/unique-identifiers.md +++ b/docs/integrations/integration-toolkit/inbound/unique-identifiers.md @@ -1,5 +1,5 @@ --- -sidebar_position: 3 +sidebar_position: 4 title: Unique Identifiers description: Configure how entities are matched and looked up --- From b4591a5808a0e3342ee92ef8e508f8238bde8f05 Mon Sep 17 00:00:00 2001 From: Karl Rankla Date: Wed, 26 Aug 2026 12:51:10 +0300 Subject: [PATCH 2/9] docs(direct-mode): render the processing flow as a mermaid chart --- .../inbound/direct-mode.md | 25 +++++++------------ 1 file changed, 9 insertions(+), 16 deletions(-) diff --git a/docs/integrations/integration-toolkit/inbound/direct-mode.md b/docs/integrations/integration-toolkit/inbound/direct-mode.md index a78d8438..c930daa1 100644 --- a/docs/integrations/integration-toolkit/inbound/direct-mode.md +++ b/docs/integrations/integration-toolkit/inbound/direct-mode.md @@ -22,22 +22,15 @@ What you give up is the mapping engine itself: field mappings, JSONata and JSONP ## How It Works -``` -Middleware sends POST /v3/erp/updates/events (payload = direct payload) - | -epilot resolves the use case configuration (direct: true) - | -The payload is validated — schema, version, entity allowlist - | (invalid events are rejected per-event, never processed) - | -The event is queued — deduplication, ordering, fairness unchanged - | -The operations are translated to internal entity / meter-reading updates - | (no mapping engine involved) - | -Create-vs-update, relations, and meter readings resolve as in mapped mode - | -Monitoring events are emitted (ENTITY_CREATED, ENTITY_UPDATED, ...) +```mermaid +flowchart TD + A["Middleware sends POST /v3/erp/updates/events
(payload = direct payload)"] --> B["epilot resolves the use case configuration
(direct: true)"] + B --> C{"Payload valid?
schema · version · entity allowlist"} + C -- no --> X["Rejected per-event, never processed
HTTP 422 + DIRECT_* monitoring event"] + C -- yes --> D["Event queued
deduplication · ordering · fairness unchanged"] + D --> E["Operations translated to internal
entity / meter-reading updates
(no mapping engine involved)"] + E --> F["Create-vs-update, relations, and meter readings
resolve as in mapped mode"] + F --> G["Monitoring events emitted
(ENTITY_CREATED, ENTITY_UPDATED, …)"] ``` ## Enabling Direct Mode From 924a7dba181cfb43e1a4bd39278e65f202640367 Mon Sep 17 00:00:00 2001 From: Karl Rankla Date: Wed, 26 Aug 2026 12:51:10 +0300 Subject: [PATCH 3/9] docs(direct-mode): unsupported list is not version-scoped No follow-up payload version is planned; the unsupported features are a property of direct mode itself, not of a "version 1". --- .../integration-toolkit/inbound/direct-mode.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/integrations/integration-toolkit/inbound/direct-mode.md b/docs/integrations/integration-toolkit/inbound/direct-mode.md index c930daa1..6de8c339 100644 --- a/docs/integrations/integration-toolkit/inbound/direct-mode.md +++ b/docs/integrations/integration-toolkit/inbound/direct-mode.md @@ -18,7 +18,7 @@ Direct mode removes only the transformation step. Everything else in the inbound - Sync activity attachment and echo prevention - Monitoring events -What you give up is the mapping engine itself: field mappings, JSONata and JSONPath expressions, constants, and the mapping-only field types (see [Unsupported in Version 1](#unsupported-in-version-1)). If your source system emits raw ERP payloads that need transformation, use [Mapping](./mapping.md) instead. A single integration can freely mix direct and mapped use cases. +What you give up is the mapping engine itself: field mappings, JSONata and JSONPath expressions, constants, and the mapping-only field types (see [Unsupported in Direct Mode](#unsupported-in-direct-mode)). If your source system emits raw ERP payloads that need transformation, use [Mapping](./mapping.md) instead. A single integration can freely mix direct and mapped use cases. ## How It Works @@ -72,7 +72,7 @@ If `entities` is **non-empty**, it acts as an allowlist for entity operations: 1. The operation's `entity_slug` must equal the `entity_schema` of some allowlist entry, otherwise the operation is rejected with `DIRECT_ENTITY_NOT_ALLOWED`. 2. The operation's `unique_ids` keys must be **exactly the declared set** (order-insensitive), **or** exactly `["_id"]` — referencing an entity directly by its epilot ID is always allowed. Any other key set is rejected with `DIRECT_ENTITY_NOT_ALLOWED`, and the error message names the expected keys. -Meter-reading operations are **not** allowlist-gated in version 1 — the allowlist applies to entity operations only. +Meter-reading operations are **not** allowlist-gated — the allowlist applies to entity operations only. ### Configuration Propagation @@ -89,7 +89,7 @@ The schema is snake_case throughout. All object schemas are **strict**: unknown The `version` field pins the payload schema: - Changes within a version are **additive-only** — existing payloads never break within version `"1"`. -- A future breaking revision arrives as version `"2"`, with both versions accepted in parallel. +- If a breaking revision is ever needed, it arrives as version `"2"` with both versions accepted in parallel — none is currently planned. - An unknown version is rejected with `DIRECT_VERSION_UNSUPPORTED`. ### Envelope @@ -461,9 +461,9 @@ Direct mode adds three monitoring codes. All three are **error**-level: Success paths reuse the existing codes — direct operations are indistinguishable from mapped ones once translated: `ENTITY_CREATED`, `ENTITY_UPDATED`, `ENTITY_DELETED`, `ENTITY_NO_OP`, `METER_READING_UPSERTED`, `METER_READING_DELETED`. -## Unsupported in Version 1 +## Unsupported in Direct Mode -The following are intentionally **out of scope** for direct mode version 1. In each case, mapped mode remains fully available — a single integration can mix direct and mapped use cases freely. +The following are intentionally **not supported** in direct mode. In each case, mapped mode remains fully available — a single integration can mix direct and mapped use cases freely. | Not supported | Why | What to use instead | |---------------|-----|---------------------| @@ -475,7 +475,7 @@ The following are intentionally **out of scope** for direct mode version 1. In e | XML payloads | The direct contract is JSON-only by design. | Send JSON; for XML-emitting sources use mapped mode. | | CSV imports against direct use cases | CSV imports emit mapping-shaped events; routed to a direct use case they fail with `DIRECT_PAYLOAD_INVALID`. | Route CSV imports to mapped use cases. | | v1/v2 events endpoints | The legacy configuration paths do not carry the `direct` flag. | `POST /v3/erp/updates/events` with `integration_id`. | -| Meter-reading allowlist gating | The entity allowlist covers entity operations only in version 1. | Open by design; gate entity operations if needed. | +| Meter-reading allowlist gating | The entity allowlist covers entity operations only. | Open by design; gate entity operations if needed. | ## Operational Notes From 13a95db78a130d01a067eb4ea09dfbcfd0a8a4a9 Mon Sep 17 00:00:00 2001 From: Karl Rankla Date: Wed, 26 Aug 2026 12:51:10 +0300 Subject: [PATCH 4/9] docs(direct-mode): lead enablement with the Direct mode UI toggle The visual configuration (toggle, Open vs Restricted write access with schema and unique-id pickers, allowlist pre-fill from mapped entities) is the primary touchpoint; the JSON is what the toggle produces. Also point direct users at simulateDirect instead of the mapped Test tab. --- .../integration-toolkit/inbound/direct-mode.md | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/docs/integrations/integration-toolkit/inbound/direct-mode.md b/docs/integrations/integration-toolkit/inbound/direct-mode.md index 6de8c339..b86aa5c3 100644 --- a/docs/integrations/integration-toolkit/inbound/direct-mode.md +++ b/docs/integrations/integration-toolkit/inbound/direct-mode.md @@ -35,7 +35,17 @@ flowchart TD ## Enabling Direct Mode -Direct mode is enabled per **inbound use case** by setting `direct: true` on the use case's `configuration`. All events routed to that use case are then interpreted as direct payloads — a use case is either fully direct or fully mapped, never both. +Direct mode is enabled per **inbound use case**. All events routed to that use case are then interpreted as direct payloads — a use case is either fully direct or fully mapped, never both. + +**In the epilot 360 UI** — the usual way to enable it: open your integration's inbound use case and +flip the **Direct mode** toggle. The JSON configuration editor disappears (direct use cases are +configured visually), and a **Write access** choice appears: **Open** (any entity may be written) or +**Restricted** (an entity allowlist built with schema and unique-id attribute pickers). Flipping the +toggle on an existing mapped use case pre-fills the allowlist from the mapped entities' schemas and +unique ids, and direct use cases are marked with a "Direct" badge in the use case overview. + +**Via the API**, the same switch is `direct: true` on the use case's `configuration` — the JSON +below is exactly what the toggle produces. The minimal configuration: @@ -377,7 +387,7 @@ An event accepted at ingest can still fail inside the pipeline in one scenario: ## Dry Run: simulateDirect -`POST /v1/erp/updates/direct_simulation` (operation ID `simulateDirect`) validates a direct payload against a configuration **without persisting anything** — the direct-mode counterpart of `simulateMappingV2`. Use it while developing your middleware, and in CI against your fixture payloads. +`POST /v1/erp/updates/direct_simulation` (operation ID `simulateDirect`) validates a direct payload against a configuration **without persisting anything** — the direct-mode counterpart of `simulateMappingV2`. Use it while developing your middleware, and in CI against your fixture payloads. (The use case **Test** tab in the epilot 360 UI applies to mapped use cases; for direct use cases, this endpoint is the dry run.) Request: From 85d3ac433801ec0ed1ca6441676926fcc9a5ae33 Mon Sep 17 00:00:00 2001 From: Karl Rankla Date: Wed, 26 Aug 2026 12:51:10 +0300 Subject: [PATCH 5/9] docs(integration-toolkit): home the config-propagation note in the configuration guide The caching window applies to every use case configuration change, so it belongs in the shared guide once instead of a standalone reminder on the direct-mode page. --- docs/integrations/integration-toolkit/configuration.md | 6 ++++++ .../integration-toolkit/inbound/direct-mode.md | 7 +------ 2 files changed, 7 insertions(+), 6 deletions(-) diff --git a/docs/integrations/integration-toolkit/configuration.md b/docs/integrations/integration-toolkit/configuration.md index 2cd44fa5..e4ae1980 100644 --- a/docs/integrations/integration-toolkit/configuration.md +++ b/docs/integrations/integration-toolkit/configuration.md @@ -139,6 +139,12 @@ curl -X PUT 'https://erp-integration.sls.epilot.io/v1/integrations/{integrationI }' ``` +:::note Configuration propagation +Use case configurations are cached by the event-processing pipeline: any configuration change — +mapping edits, enabling/disabling, mode switches — can take up to **5 minutes** to take effect for +incoming events. Plan cutovers accordingly. +::: + ### Use Case History View the change history for a use case: diff --git a/docs/integrations/integration-toolkit/inbound/direct-mode.md b/docs/integrations/integration-toolkit/inbound/direct-mode.md index b86aa5c3..02ccfe33 100644 --- a/docs/integrations/integration-toolkit/inbound/direct-mode.md +++ b/docs/integrations/integration-toolkit/inbound/direct-mode.md @@ -84,10 +84,6 @@ If `entities` is **non-empty**, it acts as an allowlist for entity operations: Meter-reading operations are **not** allowlist-gated — the allowlist applies to entity operations only. -### Configuration Propagation - -Use case configurations are cached in the pipeline for up to **5 minutes**. Plan cutovers accordingly: do not start sending direct payloads to a freshly flipped use case until the window has passed (or accept that a few events may be rejected and need to be resent). - ## Payload Contract (Version "1") The direct payload travels inside the existing `ErpEventV3.payload` — either as a JSON object or as a serialized JSON string (with `format: "json"`). XML is rejected for direct use cases. @@ -383,7 +379,7 @@ Direct payloads are validated **fully on arrival** — schema, version, and allo - Error messages are actionable: they name the operation index and the field path, e.g. `operations[3].unique_ids: at least one unique identifier is required`. - Each rejection also emits a monitoring event with the matching `DIRECT_*` code (see [Monitoring](#monitoring)). -An event accepted at ingest can still fail inside the pipeline in one scenario: the use case configuration changed between acceptance and processing (within the [5-minute cache window](#configuration-propagation)). Such failures do not retry — the event is dropped and reported via a monitoring event with the matching `DIRECT_*` code. +An event accepted at ingest can still fail inside the pipeline in one scenario: the use case configuration changed between acceptance and processing (configurations are cached — see [configuration propagation](../configuration.md#use-case-configuration)). Such failures do not retry — the event is dropped and reported via a monitoring event with the matching `DIRECT_*` code. ## Dry Run: simulateDirect @@ -491,6 +487,5 @@ The following are intentionally **not supported** in direct mode. In each case, - **Size budget.** Each event is processed as a single message with a 256 KiB limit (1 MiB for meter-reading batches). A payload holds at most **100 operations**; split larger batches across multiple events. The operation limit is additive to raise in a future revision if needed. - **No entity-attribute validation — by design.** Attributes are written verbatim; attributes not defined in the schema are stored but not indexed. This is intentional and common integration practice (mapped mode behaves the same way). The design-time guards are [simulateDirect](#dry-run-simulatedirect) and the unique-identifier schema warnings. -- **Configuration propagation.** The `direct` flag takes up to 5 minutes to reach the pipeline after a configuration change. - **Content-based deduplication.** Byte-identical events within 5 minutes collapse silently — set `deduplication_id` deliberately (see [Deduplication](#deduplication)). - **Sync activities.** A sync activity is attached to every write automatically. Direct writes do **not** echo back out through outbound delivery — the same echo prevention as mapped mode. From 76b54e66b41c09ba9ef87f0df45687add2584c83 Mon Sep 17 00:00:00 2001 From: Karl Rankla Date: Wed, 26 Aug 2026 12:51:10 +0300 Subject: [PATCH 6/9] docs(direct-mode): pin verified pipeline behaviors Facts verified against the shipped code and the dev environment: allowlist entries must declare at least one unique id; upsert restores soft-deleted matches (SOFT_DELETED_ENTITY_MATCHED); purge reports as ENTITY_DELETED with mode purge; a unique id missing from the schema still creates (plus UNIQUE_ID_NOT_IN_SCHEMA errors); _id references never create; request-level validation intercepts version/size defects on the dry-run endpoint with HTTP 400; packed meter batches emit one METER_READING_UPSERTED with the reading count; reused warning codes listed. --- .../inbound/direct-mode.md | 35 +++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/docs/integrations/integration-toolkit/inbound/direct-mode.md b/docs/integrations/integration-toolkit/inbound/direct-mode.md index 02ccfe33..5e16697a 100644 --- a/docs/integrations/integration-toolkit/inbound/direct-mode.md +++ b/docs/integrations/integration-toolkit/inbound/direct-mode.md @@ -70,6 +70,8 @@ With an optional entity allowlist: Notes: - Allowlist entries need no `fields` array — there is nothing to map. +- Every allowlist entry must declare **at least one** unique id — saving a configuration with an + empty `unique_ids` array is rejected (the UI enforces the same rule inline). - Declared `unique_ids` are checked against your real entity schemas when the use case is saved, so typos surface at design time. - Direct mode requires the **v3 events endpoint** (`POST /v3/erp/updates/events` with `integration_id`). The deprecated v1/v2 endpoints do not support it. @@ -159,6 +161,21 @@ Delete example — `attributes` may be omitted: } ``` +Behavior notes: + +- **Restore on upsert.** An upsert whose unique ids match a **soft-deleted** entity restores it + before applying the attributes (reported as `ENTITY_UPDATED`, alongside a + `SOFT_DELETED_ENTITY_MATCHED` warning). +- **Purge.** `purge` deletes irrecoverably and is reported as `ENTITY_DELETED` with mode `purge` in + the monitoring detail — there is no separate purge code. +- **Unique id not in the schema.** A unique-id attribute that does not exist in the entity schema + never matches anything — every event then **creates a new entity**, and each write also emits an + error-level `UNIQUE_ID_NOT_IN_SCHEMA` monitoring event. Catch this before go-live with + [simulateDirect](#dry-run-simulatedirect)'s schema warnings. +- **`_id` references never create.** `unique_ids: {"_id": …}` pointing at a nonexistent (or purged) + entity fails the event after retries — the `_id` form is strictly a reference to an entity that + exists. + ### Meter Reading Operations An operation with `type: "meter_reading"`: @@ -385,6 +402,15 @@ An event accepted at ingest can still fail inside the pipeline in one scenario: `POST /v1/erp/updates/direct_simulation` (operation ID `simulateDirect`) validates a direct payload against a configuration **without persisting anything** — the direct-mode counterpart of `simulateMappingV2`. Use it while developing your middleware, and in CI against your fixture payloads. (The use case **Test** tab in the epilot 360 UI applies to mapped use cases; for direct use cases, this endpoint is the dry run.) +:::note +Request-level validation intercepts some contract violations **before** the dry run executes — an +unsupported `version`, more than 100 operations, or a structurally malformed envelope return +HTTP `400` with schema errors instead of a `200` verdict. The collected-errors behavior below +applies to the checks the dry run itself performs (allowlist, relation envelopes, meter-reading +rules, unique-id values, …). On the live events endpoint the same defects surface as per-event +errors with `DIRECT_*` codes. +::: + Request: ```json @@ -467,6 +493,15 @@ Direct mode adds three monitoring codes. All three are **error**-level: Success paths reuse the existing codes — direct operations are indistinguishable from mapped ones once translated: `ENTITY_CREATED`, `ENTITY_UPDATED`, `ENTITY_DELETED`, `ENTITY_NO_OP`, `METER_READING_UPSERTED`, `METER_READING_DELETED`. +The existing pipeline warning codes also apply unchanged; the ones you are most likely to meet in +direct mode: `SOFT_DELETED_ENTITY_MATCHED` (upsert matched a soft-deleted entity — it is restored), +`UNIQUE_ID_MULTIPLE_MATCHES`, `RELATION_REF_VALUE_UNDEFINED` / `RELATION_REF_ITEM_NOT_FOUND`, and +the error-level `UNIQUE_ID_NOT_IN_SCHEMA`. + +Batch shape: several readings for the **same meter/counter in one event** are written as one batch +and produce **one** `METER_READING_UPSERTED` event whose detail carries the reading count and +external ids — not one event per reading. + ## Unsupported in Direct Mode The following are intentionally **not supported** in direct mode. In each case, mapped mode remains fully available — a single integration can mix direct and mapped use cases freely. From c09e8395a1065132270516f0ef11f201ea49b07b Mon Sep 17 00:00:00 2001 From: Karl Rankla Date: Wed, 26 Aug 2026 13:03:54 +0300 Subject: [PATCH 7/9] docs(direct-mode): link relation semantics instead of restating them Operation semantics live in the Relations page; the direct-mode page keeps only its own envelope syntax and the strict operation whitelist. --- .../integration-toolkit/inbound/direct-mode.md | 12 +++--------- 1 file changed, 3 insertions(+), 9 deletions(-) diff --git a/docs/integrations/integration-toolkit/inbound/direct-mode.md b/docs/integrations/integration-toolkit/inbound/direct-mode.md index 5e16697a..b6393d3c 100644 --- a/docs/integrations/integration-toolkit/inbound/direct-mode.md +++ b/docs/integrations/integration-toolkit/inbound/direct-mode.md @@ -227,7 +227,7 @@ Relations use the same envelope concepts as mapped mode, written directly as att ### Relation Operations -On any attribute inside `attributes`, a relation value is either a bare array (shorthand for `_set`, i.e. replace) or an object with **exactly one** operation key: +On any attribute inside `attributes`, a relation value is either a bare array (shorthand for `_set`) or an object with **exactly one** operation key — `_set`, `_append`, or `_append_all`, with the same semantics as in mapped mode (see [Relation Operations](./relations.md#relation-operations)): ```json "contacts": { "$relation": [ { "schema": "contact", "unique_ids": { "customer_number": "42" } } ] } @@ -237,13 +237,7 @@ On any attribute inside `attributes`, a relation value is either a bare array (s "contacts": { "$relation": { "_append": [ { "schema": "contact", "unique_ids": { "customer_number": "43" } } ] } } ``` -| Operation | Behavior | -|-----------|----------| -| `_set` (or bare array) | Replace the relation list. | -| `_append` | Merge into the existing list, deduplicated by entity ID. | -| `_append_all` | Merge into the existing list without deduplication. | - -**Only** `_set`, `_append`, and `_append_all` are accepted — any other key is rejected at validation time, so a typo like `_apend` fails fast with an actionable error instead of producing an unresolved relation downstream. +**Only** those three operation keys are accepted — any other key is rejected at validation time, so a typo like `_apend` fails fast with an actionable error instead of producing an unresolved relation downstream. ### Relation Items @@ -327,7 +321,7 @@ A relation reference points at an **item inside a repeatable attribute** of anot | `path` | **Yes** | Attribute on the **target** entity holding the repeatable array (e.g. `"address"`). | | `value` | **Yes** | The item to match (or create) at that path. | -The operation envelope follows the same rules as `$relation`: a bare array is shorthand for `_set`, and only `_set` / `_append` / `_append_all` are accepted. +The operation envelope follows the same rules as [`$relation`](#relation-operations). ## Sending Direct Events From f8b4e884983662b15a92b19c029a94a0463ccf43 Mon Sep 17 00:00:00 2001 From: Karl Rankla Date: Wed, 26 Aug 2026 13:08:46 +0300 Subject: [PATCH 8/9] docs(direct-mode): the Test tab runs the direct dry run in the UI simulateDirect is exposed through the use case Test tab for direct use cases, same as the mapping test for mapped ones. --- .../integrations/integration-toolkit/inbound/direct-mode.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/docs/integrations/integration-toolkit/inbound/direct-mode.md b/docs/integrations/integration-toolkit/inbound/direct-mode.md index b6393d3c..6e74f21c 100644 --- a/docs/integrations/integration-toolkit/inbound/direct-mode.md +++ b/docs/integrations/integration-toolkit/inbound/direct-mode.md @@ -394,7 +394,11 @@ An event accepted at ingest can still fail inside the pipeline in one scenario: ## Dry Run: simulateDirect -`POST /v1/erp/updates/direct_simulation` (operation ID `simulateDirect`) validates a direct payload against a configuration **without persisting anything** — the direct-mode counterpart of `simulateMappingV2`. Use it while developing your middleware, and in CI against your fixture payloads. (The use case **Test** tab in the epilot 360 UI applies to mapped use cases; for direct use cases, this endpoint is the dry run.) +`POST /v1/erp/updates/direct_simulation` (operation ID `simulateDirect`) validates a direct payload against a configuration **without persisting anything** — the direct-mode counterpart of `simulateMappingV2`. Use it while developing your middleware, and in CI against your fixture payloads. + +The same dry run is available in the epilot 360 UI: the use case's **Test** tab works for direct +use cases as it does for mapped ones — paste a payload and review the verdict, collected errors, +and translated preview without writing anything. :::note Request-level validation intercepts some contract violations **before** the dry run executes — an From b34f2592a1cc5b8c4cd59142096d65df9e3cd99e Mon Sep 17 00:00:00 2001 From: Karl Rankla Date: Wed, 26 Aug 2026 13:12:07 +0300 Subject: [PATCH 9/9] docs(direct-mode): describe sync-write behavior in business terms MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Inbound syncs do not trigger entity automations by default (a trigger opts in via its "Ignore system activities?" setting — verified against automation-api exclusion logic and the automation hub trigger defaults) and never echo back out through outbound delivery. --- docs/integrations/integration-toolkit/inbound/direct-mode.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/integrations/integration-toolkit/inbound/direct-mode.md b/docs/integrations/integration-toolkit/inbound/direct-mode.md index 6e74f21c..6efbac33 100644 --- a/docs/integrations/integration-toolkit/inbound/direct-mode.md +++ b/docs/integrations/integration-toolkit/inbound/direct-mode.md @@ -15,7 +15,7 @@ Direct mode removes only the transformation step. Everything else in the inbound - Create-vs-update resolution against existing entities - Relation and relation-reference resolution, including automatic stub creation - Meter-reading matching -- Sync activity attachment and echo prevention +- Echo prevention — inbound writes do not trigger automations by default and never flow back out - Monitoring events What you give up is the mapping engine itself: field mappings, JSONata and JSONPath expressions, constants, and the mapping-only field types (see [Unsupported in Direct Mode](#unsupported-in-direct-mode)). If your source system emits raw ERP payloads that need transformation, use [Mapping](./mapping.md) instead. A single integration can freely mix direct and mapped use cases. @@ -521,4 +521,4 @@ The following are intentionally **not supported** in direct mode. In each case, - **Size budget.** Each event is processed as a single message with a 256 KiB limit (1 MiB for meter-reading batches). A payload holds at most **100 operations**; split larger batches across multiple events. The operation limit is additive to raise in a future revision if needed. - **No entity-attribute validation — by design.** Attributes are written verbatim; attributes not defined in the schema are stored but not indexed. This is intentional and common integration practice (mapped mode behaves the same way). The design-time guards are [simulateDirect](#dry-run-simulatedirect) and the unique-identifier schema warnings. - **Content-based deduplication.** Byte-identical events within 5 minutes collapse silently — set `deduplication_id` deliberately (see [Deduplication](#deduplication)). -- **Sync activities.** A sync activity is attached to every write automatically. Direct writes do **not** echo back out through outbound delivery — the same echo prevention as mapped mode. +- **Automations and echo.** Inbound sync writes do **not** trigger entity automations by default — an automation can opt in via its trigger's "Ignore system activities?" setting — and they never echo back out through outbound delivery. Same behavior as mapped mode.