Repository navigation
v0.11.0: ContextEngine, learned JEV policy & Observatory - #19
Conversation
Verified source tree d54a0ce matches the locally tested candidate. Remove temporary offline-development transfer workflows and payloads. Prepare v0.10.0 for review only; leave main and released v0.9.0 unchanged.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Final verification — candidate remains unmerged and unpublishedVerified final head All final PR workflows completed successfully:
Downloaded and inspected the The real-Hermes smoke confirms loader discovery, ABC subclassing, factory selection, shared CLI/web options, engine-tool injection, request middleware, clone isolation, untouched full host history and exact recovery. The built-in compressor was not constructed. Its complete fixture request measured 14,531 → 4,051 tokens with two fixture JEV batches and zero paid calls. CI reproduced all committed token totals exactly: engine 31,934 / 32,100 versus quality-preserving age-collapse-off middleware 70,554 / 74,251 (GPT-4o / GPT-4 tokenizers). All 120 transformations were non-enlarging; the engine passed 30/30 visible-data golden checks. The legacy age-collapse-on arms again failed those old-evidence checks; their shorter prompts are not quality-preserving wins. CI median local latency was 590.125 / 607.052 ms for the engine and 97.384 / 117.372 ms for quality-matched middleware. These are runner-specific local times, not live service latency; the earlier committed report records the separate local-container timings. Recovery-probe full-resend input medians reproduced 6,296 / 6,324 versus 17,014 / 17,755. No real JEV/model calls were made, so live cost, real classifier/model quality and a reproducible LCM comparison remain unmeasured. Latest release independently rechecked: v0.9.0, still targeting |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0cfe572506
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if ( | ||
| "token_terminator_history" not in names | ||
| or choice == "none" | ||
| or isinstance(choice, dict) |
There was a problem hiding this comment.
When a restricted profile exposes token_terminator_history but not the separate token_terminator tool, this check permits the shared compiler/compactor pipeline to run. That pipeline can replace an old tool result with a [Token Terminator artifact ...] receipt whose recovery action requires the missing token_terminator tool, so an accepted request irreversibly hides evidence from the model. Require the general recovery tool whenever the downstream stages can emit its references, or disable those stages when it is absent.
Useful? React with 👍 / 👎.
| decision = runtime._middleware_pipeline( | ||
| request=plan.request, _tt_engine_plan=plan, **kwargs | ||
| ) |
There was a problem hiding this comment.
Calling _middleware_pipeline here writes the durable request_metrics row with plan.request as its raw payload before the engine compares the result with the actual original request. The later correction updates only token accounting and attribution, not request_metrics; consequently accepted JEV/history savings are omitted from character totals, while a candidate rejected by the outer token gate can still leave metrics describing a payload that was never sent. This corrupts status and experiment results, so the row must be rewritten from original to the accepted final payload or persistence must be deferred until after this gate.
Useful? React with 👍 / 👎.
…erve JEV chat targets
Exercise real main-request retries, auxiliary clients, embeddings, reranking, JEV-target preservation and all 72 new scope evaluations. Remove temporary transfer workflow. Keep PR #19 unmerged and v0.9.0 unchanged.
Call-scope safety review completed — latest verificationFinal review head: Already safe, preserved rather than rewrittenTT was not a global SDK/HTTP interceptor. Native Hermes auxiliary clients use separate auxiliary observer hooks; native Mem0 search/rerank/embedding integrations and SDK embedding calls do not automatically enter TT's The integration tests now exercise the real main request builder, auxiliary relay helpers, OpenAI SDK embedding serialization, and real Hermes Mem0 adapters with fake provider/HTTP endpoints. They confirm exact service arguments and prove native auxiliary clients do not dispatch TT middleware. This is a TT guarantee, not a claim that the host's own unrelated preprocessing never changes service inputs. Existing compression of service results subsequently included as conversational evidence is retained. Actual vulnerability reproduced and fixedAt original head A boundary check now runs before engine dispatch, vault/scorer/tokenizer work or model-cache mutation. Internal/auxiliary/delegated roles, non-generation operations, embedding/rerank/service envelopes, negative/unknown/conflicting purposes and unscoped generic calls veto reduction. An inherited binding or model name alone is not authorization. The registered Hermes adapter uses the actual main-hook metadata contract plus existing auxiliary-task/delegated-parent signals. Generic non-Hermes adapters explicitly authorize only real generation via out-of-band Execution-local guards prevent internal JEV/custom transport and measurement callback re-entry, including independent OpenRouter and direct TypeSafe scorer invocation, exceptions and async propagation. Known internal lifecycle calls cannot overwrite/capture into the main engine binding. Provider retries preserve that binding/cache; outgoing model changes are reconsidered each attempt. JEV as the final chat modelPurpose is decided first. Optional trustworthy host target-family metadata has priority; inspected Hermes has no dedicated target-family field, so its fallback uses the actual outgoing provider-qualified TypeSafe JEV identity, not a fuzzy substring, stale session model or TT scorer model. Custom aliases can force With room, all supplied history is retained: no semantic pruning, age-collapse, SkillGate omission or compiler receipts/working-state injection. Enabled reversible Context IR may still run if exact recovery and strict complete-request token/character savings are verified, without a JEV call just to choose IR. Only known budget pressure permits existing bounded relevance/guard/salience selection, stopping after enough safe, exactly backed omissions to fit. Output reserve and safety margin remain included; a fallback never borrows another model's window. Hard-limit qualification: protected/unscored history may still not fit. TT records Tests and final CI
Verified artifact IDs/digests:
Ordinary non-JEV regressionBoth CI jobs reproduce all original token totals exactly (five tasks counted once):
All 120 transformations per run are non-enlarging; engine visible-answer/exact-target checks remain 30/30. The two legacy age-collapse arms still fail the old-evidence checks; their short prompts are not relabelled as quality-preserving. No paid calls or new LCM comparison. README, changelog/release-note source, migration/security guidance and |
JEV architecture clarification requested by AronI rechecked TypeSafe's primary System One, coding-agent, skill-suggestion, intent-routing, RAG, citation-checking, guardrail, and verification/escalation documentation against head JEV can supply agentic control (workflow/skill/tool/handler/priority selection and escalation) and support a reasoning/writing LLM through context selection and input/output verification. These are roles composed in application code using the same typed decision primitives, not two generative modes or separate System Two API endpoints. TypeSafe explicitly says JEV itself is not a drop-in conversational/code-generation model. The current TT integration correctly uses batched Noul judgments for relevance/guard/salience and exact evidence, a subset of System 2 support. It does not implement a JEV-powered workflow controller or post-answer citation/requirements verifier. Existing deterministic SkillGate must not be described as TypeSafe's two-stage skill-suggestion integration. A preservation policy is not an implementation of those features. Code inspection found that raw Primary sources:
|
…ontext gate Document RLCD versus external policy learning and the implementation boundary. Add 18 offline regressions for feature extraction, proposal/evaluation helpers and main-request retries. No runtime, credentials, training or publication changes.
Architecture correction: external learning around JEV is not TT's fixed context gateAt Aron's clarification, checked TypeSafe's AI primer and autoresearch feature-discovery cookbook, including its implementation and validation split. These references supplement the earlier System 1 / System 2 role clarification; the earlier gate-only description did not capture the learned outer system. For reviewers and future implementers: do not conflate (a) RLCD training of JEV, (b) JEV's decision/control and LLM-support roles, and (c) training an external predictor or policy on JEV-derived semantic features. The latter can learn while JEV's weights remain unchanged. A fixed threshold, hand-written weighted sum or score cache is not evidence that this learning loop exists. The current runtime preserves continuous relevance/guard/salience Noul values but consumes them with conservative fixed rules. It does not implement an external fitted policy, CatBoost training, automated feature discovery, question revision, or a post-answer verification/escalation controller. No such capability is claimed by this change. This correctionCommit
The existing scope guard already handled these cases correctly; no runtime rewrite was needed. No new training job, dataset export, model dependency, credential, paid inference call, model fine-tuning or automatic policy promotion was introduced. README/release behavior was not changed. Final verificationAll three workflows for this commit have now completed successfully. The full Python 3.13 CI log reports 449 passed, 2 skipped, with Ruff checks and formatting passing. Python 3.10–3.12, Windows, Rust and packaging jobs also succeeded.
These are software regression checks, not validation of a learned TT policy or live JEV accuracy/economics. PR #19 remains open and unmerged. No merge, tag, release, registry publication, live wiki update or running Hermes change was performed. |
…g CI Retain PR #19 and its existing branch. Add opt-in actual-fit verification, align Cargo metadata, document the new learned layer and unreleased version, and remove temporary transfer/correction workflows. No merge, tag or publication.
AronAxe
changed the title
Windows CI exposed text-mode newline translation invalidating approval hashes. Write deterministic UTF-8 bytes on both atomic-link and exclusive-create paths; simulate CRLF translation in two platform-independent regression tests. Preserve checksum rejection and fail-open behavior.
Final v0.11.0 learned-policy verification — unmerged / unpublishedFinal head Actual implementation, not another documentation-only correctionThe new Runtime modes are off (default), shadow (return the original fixed-gate request; separately audit the learned plan), and active (approved policy adds an omission veto). Active extra questions share existing region batches. The learner cannot authorize discarding protected/uncertain material or bypass purpose checks, exact source integrity/recovery, conservative JEV-wrapper behavior or the final complete-request tokenizer gate. Standard-library-only numeric JSON inference is tested against CatBoost's exported predictions; CatBoost is training-only. SHA-256 approval, scorer/schema and evaluated generation-target identity are required. An explicit independently labelled dataset is required; no automatic vault mining, data export, self-labelling, online training or promotion is enabled. Sources sharing session/task/exact evidence are kept together across CV/holdout splits. Tests show changing holdout outcomes does not change the frozen questions/models/thresholds. Live training requires explicit consent and limits combined feature/proposal calls, rows, rounds, request volume and timeouts; these limits are not a hard dollar ceiling. Final tests and CI553 passed, 2 optional external-integration skips in the full training-enabled local suite and final learning CI artifact. This supersedes the earlier 551/102 count in the implementation-stage PR description: there are 104 new evaluations versus the 449-test baseline. All four final-head workflows are successful:
Downloaded, SHA-256 verified and inspected the final artifacts—not only green statuses. Both Hermes reports contain 135 passed / 0 failures / 0 skips, all native call-scope assertions true, and main-generation smoke 14,531 → 4,051 tokens with exact recovery and unchanged full host history. Each 120-evaluation old engine benchmark reproduces all original token totals exactly; all transformations are non-enlarging and the engine's visible-answer/target checks remain 30/30. No new LCM comparison is claimed. Cross-platform defect caught and fixedThe first Windows job exposed text-mode CRLF translation invalidating the policy approval hash. The runtime rejected the mismatch and preserved history. The final writer emits and hashes identical UTF-8 bytes on both atomic-link and exclusive-create paths. Two platform-independent regression tests simulate newline translation and both paths. Actual final Windows CI now passes; checksum enforcement was not weakened. New actual-fit synthetic benchmark160 synthetic omission experiments (128 development, 32 holdout), two feature proposals, real CatBoost fitting. The first uninformative proposal is rejected; the useful revised question is accepted. Training uses 36 fixture JEV batches plus two fixture proposer calls. The fitted policy is temporary, not shipped or automatically deployed.
All 18 deployment trials preserve protected boundaries and exact recovery, never enlarge the original full request, and make zero extra fixture calls on identical retries. Active retains all required spans while using 59.20% / 59.38% fewer tokens than full history. It is intentionally larger than the incorrectly pruned fixed result, not a claim of extra reduction versus that result. Final CI median local off/shadow/active latency: 231.897 / 327.408 / 310.514 ms (GPT-4o) and 217.912 / 369.162 / 351.991 ms (GPT-4). Deployment uses 3 fixture batches in off/active and 6 in shadow. The active induced recovery append/resend is 3,577 / 3,561 input tokens. These are runner-local times and an induced recovery probe, not live service latency or a full Hermes network transaction; the committed README separately records local-container timings. The dataset, JEV/proposer answers and recovery-cost labels are synthetic; the fixed gate was deliberately given weak scores. This proves executable learning, revision and inference wiring, not real JEV calibration, real-model answer quality, representative error rates or money savings. Zero paid API calls were made. Production activation still needs a representative independently labelled, target-specific dataset and a reviewed policy. Finite holdout eligibility is not a statistical safety certificate; existing unresolved protected-history overflow still depends on host/provider enforcement. Artifact verification
The final CI source distribution matches all 28 locally tested packaged-file hashes; wheel source contents and both archives pass release verification. Wheel SHA-256: README, configuration, security, migration, JEV roles, changelog, Python/Cargo versions, release-verifier rules and wiki/release-note sources are updated. Temporary transfer/correction workflows are absent from the final tree. Main, published releases, registries, live wiki and running Hermes remain unchanged. Ready for owner review before release. |
… dashboard Add bounded real-browser and current-Hermes backend checks. Remove temporary development/transfer workflows. Keep v0.11.0 in PR #19 only; no main, release, registry, live wiki or running Hermes changes.
AronAxe
changed the title
…ssets The installer intentionally normalizes UTF-8 text assets to LF. Test both checkout newline forms and byte-exact preservation of the user's existing CRLF configuration. No runtime or accounting change.
Dashboard completed and verified — v0.11.0 candidate remains unmergedFinal head User-facing additions
Attribution and safetyThe native integration exposed an existing profile-home precedence issue: TT could use the launch Historical shared/mismatched profile ledgers are reported as ambiguous, not reassigned to the default bot or migrated silently. Duplicate/hardlinked database mappings are rejected. Exact request identities are deduplicated; pipeline stages, turn/session counts and character-only estimates are not added to exact totals. The reader uses bounded read-only SQLite queries and cached refreshes. It returns accounting, not conversation text, artifact bodies, keys or local paths. Missing/busy/corrupt sources are marked without hiding healthy profiles. Native API configuration cannot escape the authorized profile home. Standalone binds only to Installation after installing this candidate in Hermes' Python environmenttoken-terminator install-dashboard
token-terminator dashboardOpen The Desktop counter follows the active gateway profile, not an independently focused tiled bot. Remote connections use the remote authenticated plugin API; the UI does not pretend local port 7474 belongs to a remote host. Launch/forward the dashboard on the appropriate machine when needed. Full configuration and definitions: Final verification611 passed, 2 optional external-integration skips in both the local full training-enabled suite and downloaded final learning CI report. This adds 58 dashboard/attribution tests to the earlier 553 baseline. All five final-head workflows succeeded:
Current Desktop/backend contract was inspected and exercised against Hermes Windows initially caught a test comparing CRLF checkout bytes with the installer's intentionally LF-normalized text. The final test covers both LF and CRLF resources and proves byte-for-byte preservation of an existing CRLF user config; no runtime/accounting change or assertion weakening was used. Downloaded artifact verificationAll final artifacts were SHA-256 checked and their reports inspected:
The source distribution matches all 24 locally verified packaged dashboard-file hashes; the wheel matches all 10 changed runtime/asset hashes. Both archives pass release verification. Wheel SHA-256: Temporary dashboard development/transfer workflows and payloads are absent from the final tree. Ready for owner review; still no merge or release. |
Use the complete curated release notes instead of only the last changelog addition. Upload verified SHA256SUMS alongside wheel/source; refresh the manual crate target. Remove temporary documentation workflow. Runtime, tests and Terminator artwork remain unchanged from the verified dashboard candidate.
Refresh the crates.io README and version examples without changing Rust code. Correct stale review-only wiki instructions and list all managed Desktop/backend adapter files for rollback. Preserve prior release notes and exact-evidence caveats.
AronAxe
changed the title
Released: v0.11.0 — Context, learning & observabilityOwner-approved merge and publication completed September 29, 2026.
Release presentationREADME, complete release notes, version examples, wiki navigation, migration/security and Rust guides now describe the shipped product rather than the review branch. The unshipped v0.10.0 milestone is included in v0.11.0. Two accessible self-contained SVGs show both runtime modes and the external learning loop:
The release-head distribution was compared with the prior verified dashboard candidate: all 47 runtime/source assets, 36 test files and 35 benchmark files are byte-identical. The three original Terminator artwork files ( Final verificationAll five review-head workflows completed successfully before merging:
The downloaded final learning artifact All jobs in the released commit's post-merge CI and Context IR verification passed, including Windows. Publication jobs also completed successfully: Published assets / integrityThe publication workflow built, checked with Twine/release verifier, and uploaded the wheel, source distribution and Manifest SHA-256: Boundaries unchangedThis is publication, not installation or activation. No running Hermes profile, selected engine, private vault, credentials or learned production policy was modified. Optional features remain opt-in. No paid inference tests were run. Synthetic evidence/learning results remain labelled as such; output savings remain unmeasured, subscription valuation remains API-equivalent, and unresolved protected-history overflow still needs host/provider enforcement. The Rust crate remains the interoperability companion, not a Python-runtime port. v0.9.0 and prior releases remain untouched. |
1 participant
v0.11.0 — Context, learning & observability
Owner-approved for release. Aron explicitly authorized merging this PR, publishing a fresh release, updating crates.io/wiki/README and refreshing technical schematics while preserving the Terminator artwork. The previous published version is v0.9.0; the unshipped v0.10.0 milestone is included here. The branch name retains the original review history.
What ships
Release polish
The README, changelog, install/migration/security guides and wiki now describe the complete release rather than a development branch. Two self-contained, accessible SVG schematics show the two runtime modes and the external learning loop; Terminator hero/artwork remains unchanged. Rust package guides are updated to 0.11.0 and explicitly describe the interoperability-only scope. Historical published release notes remain preserved; 0.10.0 is documented as unshipped.
docs/releases/v0.11.0.mdprovides the complete GitHub release notes. The publication workflow uses these notes and uploads verified SHA256SUMS alongside wheel/source distributions. Existing main-push workflows publish the release, Rust crate and wiki after merge. No secret values or running Hermes profiles are changed.Verification
Before this docs-only release pass, the training-enabled suite had 611 passed / 2 optional external-integration skips, including 58 dashboard/accounting tests. Main Python/Windows/Rust/package checks, real CatBoost regression, both pinned Hermes contracts, Context IR and real localhost Chromium dashboard QA passed. The final release-head workflows run the same checks again; the final results and merge/publication identities will be recorded below.
The documentation pass does not change runtime code, tests, benchmarks, defaults or Terminator artwork. No paid model calls are required.
Qualifications retained in the release
Exact counts measure complete canonical request JSON, not hidden provider billing framing. Unknown tokenizer/evidence failures fail open; protected-history overflow still needs host/provider enforcement. Persistent pins require retention planning. Archive-only recall is bounded lexical search, not universal semantic retrieval or LCM import.
Synthetic fixtures demonstrate executable learning/inference, exact recovery and token invariants, not production answer quality, service calibration, monetary saving or LCM superiority. No trained production policy ships. Output generated is not measured output saved; subscription API-equivalent value is not a reduced invoice or net saving. Desktop contribution/backend contracts and browser dashboard are tested, not a full Electron/authenticated-gateway end-to-end session.
Entry points
Release notes · ContextEngine · Learning · Dashboard
Installation remains deliberate: installing a package does not enable/select a ContextEngine, activate a learned policy, start a dashboard or modify a running profile.