feat(evidence): keep embedded image URIs as positional non-lexical units

CHANGELOG.md

@@ -6,6 +6,7 @@ All notable changes to TEPP are documented here. The format follows Keep a Chang
 
 ### Added
 
+- `evidence_core` embedded-image units: `data:image/<type>;base64,...` URIs keep their original source spans and media types, and cannot be used as lexical inference text.
 - `tepp_api` adaptive orchestration router (ADR 0010): versioned `direct`/`verify`/`committee`/`conductor`/`abstain` selection from CPU `f64` risk, ambiguity, evidence, and token-budget inputs; recorded stages, recursion, decomposition, access lists, and role-specific reasoning effort; fail-closed document-controlled policy/access/credentials; LLM plans remain proposals under deterministic statistical authority; comparable-budget ablation requires a direct baseline; credential-free contextual-orchestrator binding. Live NIM HTTP remains accepted-target.
 - `tepp_api` purpose-bound provider-payload minimization: time-bounded `PurposeGrant` evaluation, fail-closed expired/not-yet-valid/inverted/cross-tenant/impossible-calendar denial, semantic UTC calendar validation, refusal to copy identity mappings into model-provider payloads or ordinary logs, preservation of opaque analytical identifiers and membership roles (no blanket PII mask), a separately authorized scientific re-identification path, and an internally bound FIPS 180-4 SHA-256 audit digest appended through `ReidentificationAuditSink` before disclosure.
 - `persistence_postgres` backup/restore integrity: restored snapshots stay unusable until tenant, canonical `SHA-256`, knowledge-cutoff eligibility, temporal window order, and append-only triggers revalidate; SQL probes raise `restore integrity failed` (ADR 0013).

DOCUMENTATION.md

@@ -33,6 +33,7 @@ TEPP's approved PRD v0.4 and implementation plan are the primary product baselin
 | Hourly NIM product-development operations | [`docs/operations/HOURLY_NIM_PRODUCT_DEVELOPMENT.md`](docs/operations/HOURLY_NIM_PRODUCT_DEVELOPMENT.md) |
 | Actions workflow fleet audit | [`docs/operations/ACTIONS_WORKFLOW_FLEET.md`](docs/operations/ACTIONS_WORKFLOW_FLEET.md) |
 | Actions fleet research doctoring | [`docs/research/actions-workflow-fleet.md`](docs/research/actions-workflow-fleet.md) |
+| Embedded-image unit doctoring | [`docs/research/embedded-image-units.md`](docs/research/embedded-image-units.md) |
 | Retention/deletion/legal-hold doctoring | [`docs/research/retention-deletion-legal-hold.md`](docs/research/retention-deletion-legal-hold.md) |
 | Provider-payload minimization doctoring | [`docs/research/provider-payload-minimization.md`](docs/research/provider-payload-minimization.md) |
 | Adaptive orchestration router doctoring | [`docs/research/adaptive-orchestration-router.md`](docs/research/adaptive-orchestration-router.md) |

crates/evidence_core/src/error.rs

@@ -44,6 +44,8 @@ pub enum EvidenceError {
     InvalidLayoutBounds,
     /// Layout coordinates exceeded the enclosing page.
     LayoutOutOfBounds,
+    /// A base64 image data URI was treated as lexical inference text.
+    EmbeddedImageIsNotLexicalText,
 }
 
 impl fmt::Display for EvidenceError {
@@ -70,6 +72,7 @@ impl fmt::Display for EvidenceError {
             Self::InvalidPageGeometry => "page geometry must be finite and positive",
             Self::InvalidLayoutBounds => "layout bounds must be finite, nonnegative, and nonempty",
             Self::LayoutOutOfBounds => "layout bounds exceed the page geometry",
+            Self::EmbeddedImageIsNotLexicalText => "embedded image is not lexical text",
         };
         formatter.write_str(message)
     }

crates/evidence_core/src/image_unit.rs

@@ -0,0 +1,136 @@
+//! Embedded `data:image` units that keep their original source location.
+
+use crate::{DocumentRecord, EvidenceError, SourceSpan};
+
+const DATA_IMAGE_PREFIX: &str = "data:image/";
+const BASE64_MARK: &str = ";base64,";
+
+/// One embedded image located in a document body.
+#[derive(Clone, Copy, Debug, PartialEq)]
+pub struct EmbeddedImageUnit<'document> {
+    span: SourceSpan,
+    media_type: &'document str,
+}
+
+impl<'document> EmbeddedImageUnit<'document> {
+    /// Exact source span of the data URI, including the `data:image/` prefix.
+    #[must_use]
+    pub const fn span(self) -> SourceSpan {
+        self.span
+    }
+
+    /// Declared image media type (`image/png`, `image/jpeg`, …).
+    #[must_use]
+    pub const fn media_type(self) -> &'document str {
+        self.media_type
+    }
+}
+
+/// Locate `data:image/<type>;base64,...` units and retain their original spans.
+///
+/// # Errors
+///
+/// Returns [`EvidenceError::EmptySourceSpan`] when the document contains no
+/// well-formed embedded image URI.
+pub fn embedded_image_units(
+    document: &DocumentRecord,
+) -> Result<Vec<EmbeddedImageUnit<'_>>, EvidenceError> {
+    let text = document.text();
+    let mut units = Vec::new();
+    let mut search_from = 0usize;
+    while let Some(relative) = text[search_from..].find(DATA_IMAGE_PREFIX) {
+        let start = search_from + relative;
+        let after_prefix = start + DATA_IMAGE_PREFIX.len();
+        let Some(mark_rel) = text[after_prefix..].find(BASE64_MARK) else {
+            search_from = after_prefix;
+            continue;
+        };
+        let media_end = after_prefix + mark_rel;
+        let payload_start = media_end + BASE64_MARK.len();
+        let payload_end = payload_start
+            + text[payload_start..]
+                .find(|ch: char| !is_base64_payload_char(ch))
+                .unwrap_or(text.len() - payload_start);
+        if payload_end == payload_start {
+            search_from = payload_start;
+            continue;
+        }
+        // The fixed `data:image/` prefix already guarantees this media-type
+        // boundary; retaining a second prefix guard would create unreachable
+        // coverage obligations.
+        let media_type = &text[start + "data:".len()..media_end];
+        let scalar_start = text[..start].chars().count();
+        let scalar_end = scalar_start + text[start..payload_end].chars().count();
+        let span = SourceSpan::new(document, start, payload_end, scalar_start, scalar_end, None)?;
+        units.push(EmbeddedImageUnit { span, media_type });
+        search_from = payload_end;
+    }
+    if units.is_empty() {
+        return Err(EvidenceError::EmptySourceSpan);
+    }
+    Ok(units)
+}
+
+/// Refuse using a document body that still contains an embedded image as
+/// lexical inference text.
+///
+/// # Errors
+///
+/// Returns [`EvidenceError::InvalidWirePayload`] for empty input and
+/// [`EvidenceError::EmbeddedImageIsNotLexicalText`] when a `data:image`
+/// base64 URI is present.
+pub fn refuse_base64_image_as_lexical_text(text: &str) -> Result<(), EvidenceError> {
+    if text.is_empty() {
+        return Err(EvidenceError::InvalidWirePayload);
+    }
+    if text.contains(DATA_IMAGE_PREFIX) && text.contains(BASE64_MARK) {
+        return Err(EvidenceError::EmbeddedImageIsNotLexicalText);
+    }
+    Ok(())
+}
+
+fn is_base64_payload_char(ch: char) -> bool {
+    ch.is_ascii_alphanumeric() || matches!(ch, '+' | '/' | '=')
+}
+
+#[cfg(test)]
+mod tests {
+    use super::{embedded_image_units, refuse_base64_image_as_lexical_text};
+    use crate::{DocumentRecord, EvidenceError, SourceArtifact};
+
+    #[test]
+    fn jpeg_uri_and_incomplete_prefix_are_classified() {
+        let text = "x data:image/jpeg;base64,/9j/4AA= y data:image/gif y";
+        let artifact = SourceArtifact::from_bytes(text.as_bytes()).expect("artifact");
+        let document = DocumentRecord::from_text(artifact.id(), text).expect("document");
+        let units = embedded_image_units(&document).expect("jpeg");
+        assert_eq!(units.len(), 1);
+        assert_eq!(units[0].media_type(), "image/jpeg");
+        refuse_base64_image_as_lexical_text("plain note").expect("plain");
+        refuse_base64_image_as_lexical_text("data:image/png").expect("incomplete image");
+        assert_eq!(
+            refuse_base64_image_as_lexical_text("data:image/png;base64,AAAA"),
+            Err(EvidenceError::EmbeddedImageIsNotLexicalText)
+        );
+
+        let empty_text = "data:image/png;base64, following text";
+        let empty_artifact = SourceArtifact::from_bytes(empty_text.as_bytes()).expect("artifact");
+        let empty_document =
+            DocumentRecord::from_text(empty_artifact.id(), empty_text).expect("document");
+        assert_eq!(
+            embedded_image_units(&empty_document),
+            Err(EvidenceError::EmptySourceSpan)
+        );
+    }
+
+    #[test]
+    fn empty_payload_is_not_an_image_unit() {
+        let text = "data:image/png;base64,";
+        let artifact = SourceArtifact::from_bytes(text.as_bytes()).expect("artifact");
+        let document = DocumentRecord::from_text(artifact.id(), text).expect("document");
+        assert_eq!(
+            embedded_image_units(&document),
+            Err(EvidenceError::EmptySourceSpan)
+        );
+    }
+}

crates/evidence_core/src/lib.rs

@@ -7,13 +7,15 @@
 //! records, source spans whose byte, Unicode-scalar, page, and layout
 //! coordinates are validated before entering later temporal or psychometric
 //! layers, and strict versioned JSON wire contracts that reconstruct records
-//! only through the same domain validation boundary.
+//! only through the same domain validation boundary. Embedded `data:image`
+//! units keep their original offsets and are not lexical inference text.
 
 mod artifact;
 mod digest;
 mod document;
 mod error;
 mod identifier;
+mod image_unit;
 mod span;
 mod wire;
 
@@ -27,6 +29,12 @@ pub use document::DocumentRecord;
 pub use error::EvidenceError;
 /// A validated RFC 9562 `UUIDv7` evidence identifier.
 pub use identifier::EvidenceId;
+/// One embedded image located in a document body.
+pub use image_unit::EmbeddedImageUnit;
+/// Locate `data:image` base64 units with exact source spans.
+pub use image_unit::embedded_image_units;
+/// Refuse treating an embedded image URI as lexical inference text.
+pub use image_unit::refuse_base64_image_as_lexical_text;
 /// A validated page-relative location for source evidence.
 pub use span::PageLocation;
 /// An exact byte, Unicode-scalar, and optional page/layout span.

crates/evidence_core/tests/embedded_image_contract.rs

@@ -0,0 +1,53 @@
+//! Embedded base64 images keep their original location and are not lexical text.
+
+use evidence_core::{
+    DocumentRecord, EvidenceError, SourceArtifact, embedded_image_units,
+    refuse_base64_image_as_lexical_text,
+};
+
+#[test]
+fn data_uri_recovers_exact_span_and_media_type() {
+    let uri = "data:image/png;base64,iVBORw0KGgo=";
+    let text = format!("Before the figure.\n\n{uri}\n\nAfter the figure. data:image/gif y");
+    let artifact = SourceArtifact::from_bytes(text.as_bytes()).expect("artifact");
+    let document = DocumentRecord::from_text(artifact.id(), &text).expect("document");
+
+    let units = embedded_image_units(&document).expect("units");
+    assert_eq!(units.len(), 1);
+    assert_eq!(units[0].media_type(), "image/png");
+    assert_eq!(
+        &document.text()[units[0].span().byte_start()..units[0].span().byte_end()],
+        uri
+    );
+    assert_eq!(
+        refuse_base64_image_as_lexical_text(document.text()),
+        Err(EvidenceError::EmbeddedImageIsNotLexicalText)
+    );
+    refuse_base64_image_as_lexical_text("data:image/png").expect("incomplete image");
+    refuse_base64_image_as_lexical_text("Before the figure.").expect("plain text");
+    refuse_base64_image_as_lexical_text("data:image/gif y").expect("incomplete image marker");
+}
+
+#[test]
+fn documents_without_images_and_empty_payloads_fail_closed() {
+    let text = "No figures in this note.";
+    let artifact = SourceArtifact::from_bytes(text.as_bytes()).expect("artifact");
+    let document = DocumentRecord::from_text(artifact.id(), text).expect("document");
+    assert_eq!(
+        embedded_image_units(&document),
+        Err(EvidenceError::EmptySourceSpan)
+    );
+
+    assert_eq!(
+        refuse_base64_image_as_lexical_text(""),
+        Err(EvidenceError::InvalidWirePayload)
+    );
+    let empty_payload = "data:image/png;base64,";
+    let empty_artifact = SourceArtifact::from_bytes(empty_payload.as_bytes()).expect("artifact");
+    let empty_document =
+        DocumentRecord::from_text(empty_artifact.id(), empty_payload).expect("document");
+    assert_eq!(
+        embedded_image_units(&empty_document),
+        Err(EvidenceError::EmptySourceSpan)
+    );
+}

crates/evidence_core/tests/records_and_spans_contract.rs

@@ -329,6 +329,10 @@ fn every_record_validation_error_has_a_stable_message() {
             EvidenceError::LayoutOutOfBounds,
             "layout bounds exceed the page geometry",
         ),
+        (
+            EvidenceError::EmbeddedImageIsNotLexicalText,
+            "embedded image is not lexical text",
+        ),
     ];
 
     for (error, expected) in cases {

docs/LLM_ORCHESTRATION.md

@@ -1,6 +1,6 @@
 # TEPP LLM Orchestration and Test-Time Compute Contract
 
-**Status:** Partial — `tepp_api::route_orchestration` is the governed selector; live provider execution is not yet shipped.  
+**Status:** Partial — `tepp_api::route_orchestration` is the governed selector; live provider execution is not yet shipped.
 **Last reviewed:** 2026-08-13
 
 ## 1. Purpose

docs/TRACEABILITY.md

@@ -8,6 +8,7 @@ The full APA 7th standards/literature register remains `docs/research/standards-
 | Requirement / decision | Canonical basis | Source/evidence boundary | Maturity |
 |---|---|---|---|
 | immutable source evidence and exact spans | PRD; Architecture; ADR 0008 | `evidence_core`, Task 2 tests/doctoring; `persistence_postgres` source-artifact SQL insert/lookup plus idempotent retry (#40 implemented-main) | implemented-main |
+| embedded image location and non-lexical treatment | ADR 0008; research | `evidence_core` data-URI spans on the active PR | active-PR |
 | Rust numerical authority / CPU `f64` reference | ADR 0001 | current workspace foundation; future estimators | partial |
 | Rust workspace/quality foundation | ADR 0007 | workspace/CI/repository contract | implemented-main |
 | six distinct clocks and uncertain intervals | PRD; ADR 0002 | PR #8 `temporal_core` on protected main; PR #5 historical only | implemented-main |

docs/adr/0009-purpose-bound-pii-governance.md

@@ -1,9 +1,9 @@
 # ADR 0009 — Purpose-bound PII governance without blanket masking
 
-**Decision status:** Accepted  
-**Implementation maturity:** partial — persistence retention/deletion/legal-hold (migration `0007`) is implemented-main; purpose-bound provider-payload minimization (expired-purpose denial, log/source separation, separately authorized re-identification) is on the active PR and is not implemented-main until exact-head checks, review, and protected-main integration complete; deployment/provider-region evidence remains accepted-target  
+**Decision status:** Accepted
+**Implementation maturity:** partial — persistence retention/deletion/legal-hold (migration `0007`) is implemented-main; purpose-bound provider-payload minimization (expired-purpose denial, log/source separation, separately authorized re-identification) is on the active PR and is not implemented-main until exact-head checks, review, and protected-main integration complete; deployment/provider-region evidence remains accepted-target
 
-**Date:** 2026-08-10  
+**Date:** 2026-08-10
 **Supersedes:** None.
 
 ## Context

docs/adr/0010-adaptive-llm-orchestration.md

@@ -1,8 +1,8 @@
 # ADR 0010 — Adaptive LLM orchestration and test-time compute
 
-**Decision status:** Accepted  
-**Implementation maturity:** partial — `tepp_api` governed router, comparable-budget ablation record, and credential-free contextual-orchestrator binding are implemented on the active PR and are not implemented-main until exact-head checks, review, and protected-main integration complete; live NIM execution, learned conductor calibration, and production ablation evidence remain accepted-target  
-**Date:** 2026-08-10  
+**Decision status:** Accepted
+**Implementation maturity:** partial — `tepp_api` governed router, comparable-budget ablation record, and credential-free contextual-orchestrator binding are implemented on the active PR and are not implemented-main until exact-head checks, review, and protected-main integration complete; live NIM execution, learned conductor calibration, and production ablation evidence remain accepted-target
+**Date:** 2026-08-10
 **Supersedes:** The LLM orchestration-selection/ablation clauses previously co-located in ADR 0006. ADR 0006 remains authoritative for GPU/VRAM and model-credential separation; ADR 0015 governs autonomous repository-write/review/merge authority.
 
 ## Context

docs/adr/0011-standalone-modular-msa-boundary.md

@@ -1,8 +1,8 @@
 # ADR 0011 — Standalone operation and modular CWL MSA boundary
 
-**Decision status:** Accepted  
-**Implementation maturity:** partial — Rust crates are independently usable; naruon HTTP interchange (`POST /v1/analysis-runs` and `/v1/exports`, fail-closed table-access and credential headers) implemented on the active PR (not implemented-main); live HTTP service and remaining production persistence integrations remain accepted-target  
-**Date:** 2026-08-10  
+**Decision status:** Accepted
+**Implementation maturity:** partial — Rust crates are independently usable; naruon HTTP interchange (`POST /v1/analysis-runs` and `/v1/exports`, fail-closed table-access and credential headers) implemented on the active PR (not implemented-main); live HTTP service and remaining production persistence integrations remain accepted-target
+**Date:** 2026-08-10
 **Supersedes:** The broad cross-service ownership wording in ADR 0001. ADR 0001 remains authoritative for Rust-first numerical architecture.
 
 ## Context

docs/adr/0013-bitemporal-persistence-reproducibility-and-split-authority.md

@@ -1,8 +1,8 @@
 # ADR 0013 — Bitemporal persistence, reproducibility manifests, and split authority
 
-**Decision status:** Accepted  
-**Implementation maturity:** partial — migration contracts, cutoff eligibility, in-memory bitemporal adapters, live SQL session/migration port, document SQL contracts, `DATABASE_URL` SQLx gate, optional `live-sqlx` `PgPool` open/execute driver, exact-head live PostgreSQL CI, tenant RLS (`tepp_app_runtime` + session GUC), append-only reproducibility-manifest SQL insert/lookup, model-run / model-artifact / corpus-split-manifest chain (migration `0003`), append-only immutability triggers (migration `0004`), temporal interval ordering CHECK constraints (migration `0005`), typed membership-assignment storage (migration `0006`), event-relation/mention/instance SQL, source-artifact SQL, audit-event action-code validation, and concurrent document-write stress implemented-main; backup/restore integrity revalidation on the active PR  
-**Date:** 2026-08-12  
+**Decision status:** Accepted
+**Implementation maturity:** partial — migration contracts, cutoff eligibility, in-memory bitemporal adapters, live SQL session/migration port, document SQL contracts, `DATABASE_URL` SQLx gate, optional `live-sqlx` `PgPool` open/execute driver, exact-head live PostgreSQL CI, tenant RLS (`tepp_app_runtime` + session GUC), append-only reproducibility-manifest SQL insert/lookup, model-run / model-artifact / corpus-split-manifest chain (migration `0003`), append-only immutability triggers (migration `0004`), temporal interval ordering CHECK constraints (migration `0005`), typed membership-assignment storage (migration `0006`), event-relation/mention/instance SQL, source-artifact SQL, audit-event action-code validation, and concurrent document-write stress implemented-main; backup/restore integrity revalidation on the active PR
+**Date:** 2026-08-12
 **Supersedes:** None; complements ADR 0002 (temporal semantics), ADR 0008 (evidence identity), and ADR 0011 (service ownership).
 
 ## Context

docs/connectors/contextual-orchestrator-interpretation-port.md

@@ -1,6 +1,6 @@
 # contextual-orchestrator interpretation port for TEPP
 
-**Status:** Partial modular integration contract — `tepp_api::bind_contextual_orchestrator` is the credential-free binding; live HTTP remains accepted-target.  
+**Status:** Partial modular integration contract — `tepp_api::bind_contextual_orchestrator` is the credential-free binding; live HTTP remains accepted-target.
 **Last reviewed:** 2026-08-13
 
 ## Boundary

docs/connectors/naruon-artifact-consumer.md

@@ -1,6 +1,6 @@
 # naruon modular consumer contract for TEPP artifacts
 
-**Status:** Partial — versioned DTO plus HTTP interchange on the active PR; live HTTP service remaining  
+**Status:** Partial — versioned DTO plus HTTP interchange on the active PR; live HTTP service remaining
 **Last reviewed:** 2026-08-13
 
 ## Boundary