Precedent

An incident-response prototype that retrieves a recorded fix, checks whether it is allowed and executes it in a simulator. Changes can require named approval. Failed verification restores the previous state. Each run produces an evidence pack that can be checked offline.

The project began at the UK AI Agent Hackathon EP5 in July 2026. It now includes a change-gate API, browser console, permission-aware memory and incident-log analysis.

Policy before execution

A model can help describe an incident. Permission to act comes from deterministic policy and explicit approval state.

flowchart LR
    I[Incident] --> R[Retrieve permitted precedent]
    R --> P[Check policy]
    P --> H[Approval when required]
    H --> S[Snapshot and execute]
    S --> V[Verify result]
    V -->|Failure| B[Restore previous state]
    V -->|Success| M[Record verified fix]
    B --> E[Evidence pack]
    M --> E

Loading

Execution requires an exact service, error-code and target-type match. Similar wording can help retrieval but cannot authorise an action. Repeated verified successes make a class eligible for standing approval; a person must promote it. Revocation and failed verification reduce that permission.

Every executable action needs an inverse and a verification probe. The approved plan is bound to execution and its previous state is saved first. This supports reversible simulator operations; external services would need their own recovery design.

Implementation

Area	Code and purpose
Gate API	Service and models validate proposals and return deny, needs-approval or allow-standing. Expired decision references cannot execute.
Execution	Orchestrator and tools bind plans, take snapshots and run verification or rollback.
Memory access	Retrieval requires access to every source behind a derived fix. Missing or stale restricted permissions deny access.
Evidence	Builder and verifier check audit links and bind displayed decisions to recorded events.
Browser console	Read API checks current access when exporting cases. Sessions isolate browser simulation worlds.
Log analysis	Analyzer separates information available at incident opening from later outcomes.

Evaluation

The permission benchmark matched an independent reference on 10,000 seeded queries, including 5,219 denials and 4,781 allowances. This checks the policy implementation on a constructed corpus. It does not estimate production data-leak risk.

The incident-log analysis was corrected to require previous tickets to be resolved before the next incident opened. It uses first-recorded symptom fields to avoid later information leaking into the comparison. Under that chronology, 96.5% of 24,849 symptom-labelled incidents had a matching class already resolved. That measures recurrence, not whether the same fix would work.

The methodology, script and results record the cohorts and input hash. Historical analysis remains in the hackathon archive with a correction notice. The old top-k predictor has not been revalidated.

Run and inspect

Use Python 3.13 and uv.

make install
make sim
# Open http://localhost:8000/demo for the guided demo
# Open http://localhost:8000/console for the case-file console

No API keys are needed for the local simulator. It uses committed seed data, model fallbacks and a local Jira-shaped ACL source. Optional live model and Jira settings are described in .env.example; live integrations are not necessary for verifying the deterministic core. Model identifiers are centralized in precedent/models.py.

The default install uses uv.lock and includes core/dev dependencies. The container installs the hash-pinned runtime export in requirements-runtime.txt. For the optional agent rails:

uv sync --locked --extra dev --extra agents

Analyze an incident export locally:

.venv/bin/precedent-analyze tests/fixtures/jira_sample.csv
# Map differently named columns with --map; use --help for output options.

The analyzer does not open network sockets. It reports missing dates and assumptions about the fields in flat exports. For the UCI reproduction, download the source CSV and run the documented command in METHODOLOGY.md.

Verify

make test
make lint check-open-weight check-product-imports copy-lint policy-lint
make evidence-pack
make bench bench-extractor
make secrets-scan            # requires gitleaks

make evidence-pack executes seeded cases and verifies their JSON packs with the stdlib-only verifier. Browser tests additionally require the e2e extra and installed Chromium. Optional suites are not collected when their dependencies are absent; UCI corpus tests need PRECEDENT_UCI_CSV=/path/to/incident_event_log.csv.

uv sync --locked --extra dev --extra agents --extra e2e
.venv/bin/playwright install chromium
.venv/bin/python -m pytest tests/e2e -q

The browser regression covers approval, revocation, rollback and evidence inspection. Selecting another incident clears stale controls immediately; superseded responses cannot repopulate the old case's ladder. Session tests exercise independent worlds and process-private cold-start templates. These checks do not replace load testing or a security review of a production identity and deployment integration.

Scope

The hosted application uses isolated simulator worlds and demo principal names. Connecting real systems requires authenticated identities, protected control routes and durable shared storage.

The evidence hash chain detects inconsistent edits relative to its stored anchors. A trusted signature or independently stored anchor would be needed to authenticate a whole bundle. Restricted exports currently fail if a complete chain would expose another incident's content.

Data provenance records catalogues, datasets and adapted runbooks. Historical integration records describe past exercises. Project code uses the MIT licence; third-party material retains its own terms.