additional lang support and improvements

[pkieltyka]

Summary

Adds first-class Rust, Python, Solidity, and JavaScript support across Codegenie's review pipeline, then fixes two pipeline defects that made large reviews unreliable and skill quality unmeasurable.

Language support is end-to-end rather than file-extension detection alone: each language receives syntax-aware repository context, deterministic likely-test discovery, dedicated review guidance, isolated lens projection, packaged grammar support, and production-path validation.

On top of that, Plan 100 stops large PRs from silently discarding the entire planner output, and Plan 101 makes Stage 9 verify each candidate with the exact skill set that produced it — a prerequisite for trusting any future skill comparison. The PR also adds standalone PR-head CI and completes the Plan 99 correctness follow-up.

Multi-language support (Plan 98)

• Added tree-sitter adapters for Rust, Python, and Solidity, while retaining the shared ECMAScript parser implementation for TypeScript, TSX, and JavaScript.
• Added dedicated default-enabled bundled skills for lang/rust, lang/python, lang/solidity, and lang/javascript.
• Narrowed lang/typescript to TypeScript/TSX so JavaScript no longer receives TypeScript-only guidance.
• Established canonical language identity across routing, classification, parsing, packet construction, planning, Stage 7 review, and Stage 9 verification.
• Added syntax-aware symbols, enclosing declarations, imports, bounded signatures, ownership context, and stable declaration identities.
• Added deterministic language-specific likely-test conventions plus a generic exact-stem fallback under test, tests, and __tests__ for other languages.
• Enforced language-gated lens/skill projection so packets and prompts exclude unrelated language guidance.
• Added structural fixtures, integration coverage, positive eval cases, and marker-free negative controls for all four language slices.
• Expanded the packed-consumer smoke test to load all bundled skills and resolve/parse all seven grammar assets outside the source checkout.
• Added standalone PR-head CI with SHA256-verified actionlint, pinned pnpm, frozen scripts-enabled installation, typechecking, tests, and package builds.
• Fixed dependency repacking in the package smoke test by suppressing dependency lifecycle scripts while retaining the approved scripts for the actual consumer installation.
• Updated the functional, architecture, component, planning, packaging, and release documentation.

Rust

Supports attribute-aware functions, structs, unions, enums, traits, implementations, methods, modules, constants, macros, imports, and contextual impl ownership. Blanket impls use a deterministic impl target owner rather than treating a type parameter as nominal. Likely tests cover sibling <stem>_test.rs and nearest Cargo-package tests/<stem>.rs conventions.

Python

Supports decorated and multiline functions/classes, async functions and methods, nested ownership, imports, colon-bounded signatures, and common test_*.py / *_test.py conventions. Signatures and classification remain correct when non-ASCII or astral Unicode precedes a declaration.

Solidity

Supports contracts, abstract contracts, interfaces, libraries, functions, modifiers, constructors, fallback/receive handlers, events, errors, types, contract-owned state values/constants, imports, overload-safe identity, and deterministic default Foundry test discovery.

JavaScript

Provides distinct JavaScript language identity and lang/javascript guidance for .js, .jsx, .mjs, and .cjs. Coverage includes functions, generators, arrow/function expressions, classes and class expressions, callable fields, JSX, ESM imports/re-exports, literal CommonJS require, bounded signatures, and JavaScript test conventions without admitting TypeScript-only syntax or checks.

Language correctness follow-up (Plan 99)

• Replaced five UTF-8 Buffer slices with node-relative JavaScript string slices matching web-tree-sitter's UTF-16 coordinate space.
• Added astral-Unicode regressions for Python signatures/async classification and changed-symbol identity, Rust grouped imports/macros, and Solidity signatures.
• Restored generic exact-stem test-directory discovery without reintroducing cross-language Go/TypeScript sibling matches.
• Corrected Rust blanket-impl ownership, removed an obsolete grammar adapter and dead Solidity branch, and simplified Rust test-path classification.

Planner coverage survival and dispatch resilience (Plan 100)

Two production runs against a 90-file / 213-hunk PR reproduced total plan loss: the planner produced an accurate risk map, but its coverage entries were dropped as unknown hunk IDs, coverage ended empty, dispatch degraded to path-alphabetical order, and the runtime budget expired with the entire risk core unreviewed — reporting "No credible findings" for a money-path PR.

The root cause was that full-length hunk hashes were too expensive to keep path-associated inside a compacted planner dossier, so compaction discarded the association and the model addressed its targets by path instead.

• Introduced short hunk IDs via two-pass allocation over the complete parsed diff, so path association stays affordable and unconditional.
• Preserved planner coverage through delivery instead of silently discarding unmatched entries.
• Prioritized dispatch so planner-identified risk is reviewed before alphabetically-early trivia when the budget is finite.
• Surfaced plan-loss conditions rather than proceeding as an empty plan with warn-level noise only.
• Added semantic hunk/packet/candidate identity-parity artifacts proving a bijective mapping and exact parity across the ID migration.

Exact skill provenance and bundled-skill revision (Plan 101)

Stage 9 previously reloaded skills from the candidate's primary lens. A language-primary packet therefore received language plus both core skills at Stage 7, but only the language skill's false-positive rules at verification. That made any skill edit unmeasurable: a change in final findings could come from the verifier seeing a different rule set rather than from the edited skill.

Provenance foundation

• Added projectedSkillIds(), deriving ordered unique ids from the actual prompt projection; per-skill-truncated entries count, total-cap-omitted entries do not.
• Stamped exact per-pass ids on Stage-7 findings, follow-up hints, and uncertainties; on Stage-8 system findings; and inherited the selected source item's list in uncertainty promotion. Ensemble/adaptive pooling preserves item-level provenance instead of requiring cross-pass equality or unioning lists.
• Stage 9 now resolves guidance only through skillsById(candidate.producedBy.skillIds) plus a defensive language filter. Unknown and language-incompatible ids are dropped and reported, lensId is never a selection input, empty provenance is authoritative and omits the entire guidance block (p9.6), and missing provenance is treated as malformed rather than inferred.
• Added verifier_skill_provenance telemetry with requested/resolved/dropped/unknown/language-incompatible ids and no repository content.

Bundled-skill content

• Standardized every language check on the five-field owner matrix (failure, materiality, unsafe, safe, mitigation) and removed the meta-only Examples sections; the two core skills keep theirs because they hold distinct worked cases.
• Made False Positives and Safe Patterns readable standalone, since Stage 9 projects them without Checks.
• Corrected objective mismatches: JavaScript module interop, Python mutable-default/None/exact-unit/subprocess/TOCTOU examples, and Solidity oracle validation (no deprecated answeredInRound).
• Fixed a silent Solidity Stage-8 truncation. Every bundled skill and every canonical language-plus-core composite now projects untruncated at Stages 7, 8, and 9, enforced by owner tests.
• No check was added, removed, or merged. Inventory changes remain gated on the semantic evidence protocol below.

Semantic skill evals

• Added marker-free TypeScript, Python, and Solidity fixtures under evals/skill-semantics/, separate from the fake transport suite. Each proves a passing base, buildable feature, exact reproducible positive failure, and passing safe control against pinned toolchains, with pre-registered three-draw model arms.

Operational changes

• Made review runtime configurable as review.maxTime in minutes (--max-time still overrides per run), with an explicit supported maximum and long-timeout handling. This replaces the previous internal review.timeoutMs key; since the config schema is strict, a codegenie.toml that set the old undocumented key must switch to maxTime.
• Raised the default subagent concurrency from 4 to 6.
• Upgraded the pinned pi dependency.
• Added the Plan 102 same-file packet packing plan. Planning only — no implementation in this PR.

Validation

• pnpm run check: passing
• Full test suite: 761/761 passing across 40 files
• pnpm build: passing
• Packed external-consumer installation: passing, including all bundled skills and seven grammars
• Public fake/eval fixture suite: passing, including marker-free negative controls
• Semantic skill fixtures: passing — deterministic base-pass, feature-build, exact positive failure, and safe-control proofs for TypeScript, Python, and Solidity
• Non-posting Rust and JavaScript owner-acceptance smokes: passing

Compatibility and measurement notes

The new default-enabled skills and the JavaScript/TypeScript guidance split intentionally change the Stage 5 inventory/content, LensRegistry.registryHash(), and model-call cache identity. Branch development exposed two recorded language measurement boundaries: the Rust/Python/Solidity skill landing, followed by the atomic JavaScript skill plus TypeScript narrowing. Measurements crossing either boundary are not directly comparable prompt/cache regimes.

Plan 99 changes deterministic repository context only and adds no further skill, prompt-template, registry-hash, dependency, workflow, review-budget, provider, or LLM response-schema boundary.

Plan 100 changes hunk-ID form and planner/dispatch behavior; the identity-parity artifacts pin exact semantic parity across that migration.

Plan 101 establishes a further boundary. The bundled-skill revision rotates every affected contentSha and the registry hash, and the foundation commit bundles both corrected Stage-9 input selection and the p9.6 prompt-construction change — later prompt-byte comparisons must not attribute all Stage-9 differences to provenance alone. The paid current-skill baseline, isolated candidate arms, and integration non-inferiority comparison are not run and pending explicit authorization, so the language-skill revisions in this PR are candidate-arm content rather than accepted measurement evidence. The open real-repository eval-diversity guard remains open: the new semantic fixtures are synthetic and do not close it.

[github-actions]

codegenie review

Sorry, this review is incomplete. The allotted max time of 30 minutes was reached and the review has been degraded. Re-run with --max-time <minutes> (config review.timeoutMs, or review.maxTimeMinutes in eval cases) for a higher time allotment.

No credible findings.

Coverage

Partial review: 486 hunks were not reviewed because budget was exhausted before dispatch.
Reviewed 40/550 hunks before stopping.
Incomplete work: skipped 24, failed 486, verification incomplete 1.
Coverage levels: deep 57, normal 446, light 23, skip 24.
Unreviewed hunks by file:

• .github/workflows/ci.yml: 1 hunk (budget stopped before dispatch)
• action.yml: 1 hunk (budget stopped before dispatch)
• bundled-skills/core/code-review.md: 1 hunk (budget stopped before dispatch)
• bundled-skills/lang/go.md: 1 hunk (budget stopped before dispatch)
• bundled-skills/lang/javascript.md: 1 hunk (budget stopped before dispatch)
• bundled-skills/lang/python.md: 1 hunk (budget stopped before dispatch)
• bundled-skills/lang/rust.md: 1 hunk (budget stopped before dispatch)
• bundled-skills/lang/solidity.md: 1 hunk (budget stopped before dispatch)
• bundled-skills/lang/typescript.md: 1 hunk (budget stopped before dispatch)
• evals/fixtures/artifacts/plan-100-hunk-id-parity/README.md: 1 hunk (budget stopped before dispatch)
• evals/fixtures/javascript.yml: 1 hunk (budget stopped before dispatch)
• evals/fixtures/python.yml: 1 hunk (budget stopped before dispatch)
• 142 additional files omitted.
• Budget stopped review work (runtime reserve reached).
• Verification incomplete for 1 candidate.
• pnpm-lock.yaml: lockfile
• semantic composition skipped; deterministic fallback used

Needs Human Attention

• Do the values in DossierHunkIndexEntry.language (from factsByPath language / file.language) exactly match the strings in LensDescriptor.languages, or can normalization mismatches (e.g. hunk language 'tsx'/'ts' vs lens languages ['typescript']) cause the language-lens guard at planner.ts:1393-1406 to drop a valid language lens while core lenses survive (so defaultedLenses stays false and no fallback restores it)?
  Files: src/pipeline/planner.ts, src/skills/lens-registry.ts
  Symbols: DossierHunkIndexEntry, buildLensRegistry, defaultLensesForFile, validatePlan
  Reason: The new guard uses strict descriptor.languages.includes(hunkLanguage), while defaultLensesForFile uses fuzzy ts/tsx/typescript matching. If hunk language strings differ from lens language strings, a language-specific lens can be silently dropped for its own hunk whenever at least one core lens survives (defaultedLenses=false skips the fallback). Confirm the language vocabularies are identical to rule out lost language coverage.

Stats

Model: anthropic claude-opus-4-8 high
Elapsed time: 30m 28s
Git: 0xPolygon/codegenie from master to next (298190d)
Review completeness: partial.
Usage: model calls 81, tokens 2153873, cost $10.2350.
Effective caps: tokens 8000000.
Budget dispatch blocks: stage 7 runtime 1620822/1800000; stage 7 runtime 1632936/1800000; stage 7 runtime 1640497/1800000; stage 7 runtime 1644097/1800000; stage 7 runtime 1650542/1800000; stage 9 runtime 1822906/1800000; stage 10 runtime 1827994/1800000.
Local context pressure: 6 tool-budget rejections, 178 degraded tool results, 309 degraded hunks.

No Findings

No credible findings were found.

— workflow run