Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@ boundaries above remain the target modular MSA architecture.
| `tepp_simulation` | known-truth temporal/event data generation |
| `validation_core` | RMSE, bias, coverage, graph, and Monte Carlo metrics |
| `tepp_api` | versioned DTO, schema, and export contracts |
| `copied_text` | copied-text residue is not unique latent content and not stopword deletion |

No crate exposes placeholder production behavior in Task 1. This prevents an
empty façade from becoming a de facto public API before its invariants and tests
Expand Down
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ All notable changes to TEPP are documented here. The format follows Keep a Chang

### Added

- `copied_text` identity gate: copied and boilerplate residue is not unique latent content and is not erased by a stopword list; recovered copied-text kinds match known truth at a higher computed rate than collapsing every token to unique content (ADR 0004/0012).
- `tepp_api` adaptive orchestration router (ADR 0010): versioned `direct`/`verify`/`committee`/`conductor`/`abstain` selection from CPU `f64` risk, ambiguity, evidence, and token-budget inputs; recorded stages, recursion, decomposition, access lists, and role-specific reasoning effort; fail-closed document-controlled policy/access/credentials; LLM plans remain proposals under deterministic statistical authority; comparable-budget ablation requires a direct baseline; credential-free contextual-orchestrator binding. Live NIM HTTP remains accepted-target.
- `tepp_api` purpose-bound provider-payload minimization: time-bounded `PurposeGrant` evaluation, fail-closed expired/not-yet-valid/inverted/cross-tenant/impossible-calendar denial, semantic UTC calendar validation, refusal to copy identity mappings into model-provider payloads or ordinary logs, preservation of opaque analytical identifiers and membership roles (no blanket PII mask), a separately authorized scientific re-identification path, and an internally bound FIPS 180-4 SHA-256 audit digest appended through `ReidentificationAuditSink` before disclosure.
- `persistence_postgres` backup/restore integrity: restored snapshots stay unusable until tenant, canonical `SHA-256`, knowledge-cutoff eligibility, temporal window order, and append-only triggers revalidate; SQL probes raise `restore integrity failed` (ADR 0013).
Expand Down
4 changes: 4 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ members = [
"crates/tepp_simulation",
"crates/validation_core",
"crates/tepp_api",
"crates/copied_text",
]
default-members = [
"crates/evidence_core",
Expand All @@ -23,6 +24,7 @@ default-members = [
"crates/tepp_simulation",
"crates/validation_core",
"crates/tepp_api",
"crates/copied_text",
]

[workspace.package]
Expand Down
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ implemented in Rust.
## Current implementation state

This branch establishes the Task 1 Rust workspace and quality-gate foundation.
The ten bounded crates compile independently but intentionally expose no
The eleven bounded crates compile independently but intentionally expose no
placeholder production APIs. Domain behavior begins in Task 2 with immutable
evidence identifiers and source records.

Expand All @@ -22,6 +22,7 @@ crates/corpus_split
crates/tepp_simulation
crates/validation_core
crates/tepp_api
crates/copied_text
```

## Local verification
Expand Down
17 changes: 17 additions & 0 deletions crates/copied_text/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
[package]
name = "copied_text"
description = "Copied-text residue is not unique content and not stopword deletion."
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
authors.workspace = true
repository.workspace = true
homepage.workspace = true
readme.workspace = true
keywords.workspace = true
categories.workspace = true
publish = false

[lints]
workspace = true
55 changes: 55 additions & 0 deletions crates/copied_text/src/error.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
//! Fail-closed copied-text errors.

use std::fmt;

/// A fail-closed copied-text error.
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
#[non_exhaustive]
pub enum CopiedTextError {
/// Copied-text residue was treated as unique latent content.
CopiedTextIsNotUniqueContent,
/// Copied-text residue was treated as stopword deletion.
CopiedTextIsNotStopwordDeletion,
/// A recovery slice was empty or length-mismatched.
InvalidCopiedPayload,
}

impl fmt::Display for CopiedTextError {
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
let message = match self {
Self::CopiedTextIsNotUniqueContent => {
"copied-text residue is not unique latent content"
}
Self::CopiedTextIsNotStopwordDeletion => "copied-text residue is not stopword deletion",
Self::InvalidCopiedPayload => "invalid copied-text payload",
};
formatter.write_str(message)
}
}

impl std::error::Error for CopiedTextError {}

#[cfg(test)]
mod tests {
use super::CopiedTextError;

#[test]
fn error_messages_are_stable() {
for (error, message) in [
(
CopiedTextError::CopiedTextIsNotUniqueContent,
"copied-text residue is not unique latent content",
),
(
CopiedTextError::CopiedTextIsNotStopwordDeletion,
"copied-text residue is not stopword deletion",
),
(
CopiedTextError::InvalidCopiedPayload,
"invalid copied-text payload",
),
] {
assert_eq!(error.to_string(), message);
}
}
}
128 changes: 128 additions & 0 deletions crates/copied_text/src/kind.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
//! Copied-text residue versus unique latent content.

use crate::CopiedTextError;

/// Closed vocabulary of copied-text token treatments.
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub enum CopiedKind {
/// Copied or boilerplate residue, not unique document meaning.
CopiedText,
/// Token treatment reserved for unique latent content.
UniqueContent,
}

impl CopiedKind {
/// Return the stable wire kind name.
#[must_use]
pub const fn wire_name(self) -> &'static str {
match self {
Self::CopiedText => "copied_text",
Self::UniqueContent => "unique_content",
}
}

/// Parse a stable wire kind name.
///
/// # Errors
///
/// Returns [`CopiedTextError::InvalidCopiedPayload`] for unrecognized names.
pub fn from_wire_name(name: &str) -> Result<Self, CopiedTextError> {
match name {
"copied_text" => Ok(Self::CopiedText),
"unique_content" => Ok(Self::UniqueContent),
_ => Err(CopiedTextError::InvalidCopiedPayload),
}
}
}

/// Refuse to treat copied-text residue as unique latent content.
///
/// # Errors
///
/// Returns [`CopiedTextError::CopiedTextIsNotUniqueContent`] when `kind` is
/// [`CopiedKind::CopiedText`].
pub fn refuse_copied_text_as_unique_content(kind: CopiedKind) -> Result<(), CopiedTextError> {
match kind {
CopiedKind::CopiedText => Err(CopiedTextError::CopiedTextIsNotUniqueContent),
CopiedKind::UniqueContent => Ok(()),
}
}

/// Refuse to treat copied-text residue as stopword deletion.
///
/// # Errors
///
/// Returns [`CopiedTextError::CopiedTextIsNotStopwordDeletion`] when `kind` is
/// [`CopiedKind::CopiedText`].
pub fn refuse_copied_text_as_stopword_deletion(kind: CopiedKind) -> Result<(), CopiedTextError> {
match kind {
CopiedKind::CopiedText => Err(CopiedTextError::CopiedTextIsNotStopwordDeletion),
CopiedKind::UniqueContent => Ok(()),
}
}

/// Fraction of recovered copied-text kinds that match known truth.
///
/// # Errors
///
/// Returns [`CopiedTextError::InvalidCopiedPayload`] when either slice is
/// empty or the lengths differ.
pub fn identity_recovery_rate(
truth: &[CopiedKind],
decided: &[CopiedKind],
) -> Result<f64, CopiedTextError> {
if truth.is_empty() || truth.len() != decided.len() {
return Err(CopiedTextError::InvalidCopiedPayload);
}
let mut matches = 0_u32;
for (truth_kind, decided_kind) in truth.iter().zip(decided) {
if truth_kind == decided_kind {
matches += 1;
}
}
Ok(f64::from(matches) / truth.len() as f64)
}

#[cfg(test)]
mod tests {
use super::{
CopiedKind, identity_recovery_rate, refuse_copied_text_as_stopword_deletion,
refuse_copied_text_as_unique_content,
};
use crate::CopiedTextError;

#[test]
fn local_branches_cover_kinds_payloads_and_wire_names() {
assert_eq!(
refuse_copied_text_as_unique_content(CopiedKind::CopiedText),
Err(CopiedTextError::CopiedTextIsNotUniqueContent)
);
assert_eq!(
refuse_copied_text_as_stopword_deletion(CopiedKind::CopiedText),
Err(CopiedTextError::CopiedTextIsNotStopwordDeletion)
);
refuse_copied_text_as_unique_content(CopiedKind::UniqueContent).expect("unique");
refuse_copied_text_as_stopword_deletion(CopiedKind::UniqueContent).expect("unique");
for kind in [CopiedKind::CopiedText, CopiedKind::UniqueContent] {
assert_eq!(
CopiedKind::from_wire_name(kind.wire_name()).expect("round-trip"),
kind
);
}
assert_eq!(
CopiedKind::from_wire_name("stopword"),
Err(CopiedTextError::InvalidCopiedPayload)
);
let matched = identity_recovery_rate(&[CopiedKind::CopiedText], &[CopiedKind::CopiedText])
.expect("rate");
assert!((matched - 1.0).abs() < f64::EPSILON);
assert_eq!(
identity_recovery_rate(&[], &[]),
Err(CopiedTextError::InvalidCopiedPayload)
);
assert_eq!(
identity_recovery_rate(&[CopiedKind::CopiedText], &[]),
Err(CopiedTextError::InvalidCopiedPayload)
);
}
}
22 changes: 22 additions & 0 deletions crates/copied_text/src/lib.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
#![forbid(unsafe_code)]
#![deny(missing_docs)]
#![allow(clippy::cast_precision_loss)]
//! Copied-text residue is not unique latent content.
//!
//! Copied and boilerplate passages stay explicit method/background structure.
//! They are not unique document meaning and are not erased by a stopword list
//! (ADR 0004/0012).

mod error;
mod kind;

/// Fail-closed copied-text errors.
pub use error::CopiedTextError;
/// Closed vocabulary of copied-text token treatments.
pub use kind::CopiedKind;
/// Fraction of recovered copied-text kinds that match known truth.
pub use kind::identity_recovery_rate;
/// Refuse to treat copied-text residue as stopword deletion.
pub use kind::refuse_copied_text_as_stopword_deletion;
/// Refuse to treat copied-text residue as unique latent content.
pub use kind::refuse_copied_text_as_unique_content;
67 changes: 67 additions & 0 deletions crates/copied_text/tests/copied_text_contract.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
//! Copied-text residue is not unique content and not stopword deletion.

use copied_text::{
CopiedKind, CopiedTextError, identity_recovery_rate, refuse_copied_text_as_stopword_deletion,
refuse_copied_text_as_unique_content,
};

#[test]
fn copied_text_cannot_become_unique_content_or_stopword_deletion() {
assert_eq!(
refuse_copied_text_as_unique_content(CopiedKind::CopiedText),
Err(CopiedTextError::CopiedTextIsNotUniqueContent)
);
assert_eq!(
refuse_copied_text_as_stopword_deletion(CopiedKind::CopiedText),
Err(CopiedTextError::CopiedTextIsNotStopwordDeletion)
);
refuse_copied_text_as_unique_content(CopiedKind::UniqueContent).expect("unique");
refuse_copied_text_as_stopword_deletion(CopiedKind::UniqueContent).expect("unique");
}

#[test]
fn recovered_kinds_match_known_truth_better_than_a_unique_content_collapse() {
let truth = [
CopiedKind::CopiedText,
CopiedKind::UniqueContent,
CopiedKind::CopiedText,
];
let recovered = truth;
let collapsed = [
CopiedKind::UniqueContent,
CopiedKind::UniqueContent,
CopiedKind::UniqueContent,
];
let recovered_rate = identity_recovery_rate(&truth, &recovered).expect("recovered");
let collapsed_rate = identity_recovery_rate(&truth, &collapsed).expect("collapsed");
let expected = {
let mut matches = 0_u32;
for (truth_kind, decided_kind) in truth.iter().zip(recovered.iter()) {
if truth_kind == decided_kind {
matches += 1;
}
}
f64::from(matches) / f64::from(u32::try_from(truth.len()).expect("len"))
};
assert!((recovered_rate - expected).abs() < f64::EPSILON);
assert!(recovered_rate > collapsed_rate);
}

#[test]
fn empty_or_mismatched_kind_payloads_fail_closed() {
assert_eq!(
identity_recovery_rate(&[], &[]),
Err(CopiedTextError::InvalidCopiedPayload)
);
assert_eq!(
identity_recovery_rate(&[CopiedKind::CopiedText], &[]),
Err(CopiedTextError::InvalidCopiedPayload)
);
assert_eq!(
identity_recovery_rate(
&[CopiedKind::CopiedText, CopiedKind::UniqueContent],
&[CopiedKind::CopiedText]
),
Err(CopiedTextError::InvalidCopiedPayload)
);
}
7 changes: 7 additions & 0 deletions crates/copied_text/tests/crate_contract.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
//! Integration contract for the `copied_text` package identity.

#[test]
fn package_identity_is_stable() {
let observed = std::hint::black_box(env!("CARGO_PKG_NAME"));
assert_eq!(observed, "copied_text");
}
2 changes: 1 addition & 1 deletion docs/TRACEABILITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ The full APA 7th standards/literature register remains `docs/research/standards-
| TRSL-TM temporal/relational topic posterior and backend compatibility | ADR 0012; ADR 0004 | future `topic_measurement` | accepted-target |
| global P0 topic identity with activity/dormancy/reactivation | ADR 0012 | future topic lineage/activity state | accepted-target |
| no default stopword deletion / no TF-IDF-BM25 inferential weighting | ADR 0004/0012; PRD/TRD | future semantic/method-source model | accepted-target |
| report template/section/copied/style/modality method effects | ADR 0004/0012; PRD/TRD | simulation truth factors implemented; estimator-side method model remains future | partial |
| report template/section/copied/style/modality method effects | ADR 0004/0012; PRD/TRD | simulation truth factors implemented; `copied_text` copied-versus-unique-content identity on the active PR; estimator-side method model remains future | partial |
| candidate K statistical/Pareto gates + blinded LLM review | ADR 0012; research | future `model_selection` | accepted-target |
| compositional topic correlation / stable clustering | ADR 0005/0012; research | future `network_analysis` | accepted-target |
| posterior ESEM / longitudinal invariance / DSEM | ADR 0005 | future `psychometric_core` | accepted-target |
Expand Down
2 changes: 1 addition & 1 deletion docs/adr/0004-shared-multilingual-latent-space.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# ADR 0004 — Shared multilingual latent semantic space

**Decision status:** Accepted
**Implementation maturity:** accepted-target
**Implementation maturity:** accepted-target — copied-versus-unique-content identity in `copied_text` on the active PR; shared-space estimators remain accepted-target
**Date:** 2026-08-05
**Supersedes:** None. ADR 0012 governs the complete topic-estimator/backend/global-topic contract built on this multilingual measurement decision.

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# ADR 0012 — Temporal Relational Shared-Latent Topic Measurement

**Decision status:** Accepted
**Implementation maturity:** accepted-target
**Implementation maturity:** accepted-target — copied-versus-unique-content identity in `copied_text` on the active PR; estimator-side method model remains accepted-target
**Date:** 2026-08-12
**Supersedes:** None; refines ADR 0004 and ADR 0005 without replacing their multilingual and psychometric authorities.

Expand Down
Loading
Loading