Repository navigation
feat(gate): fail the rule, ledger, and owned blind spots - #40
Merged
Merged
Conversation
No omp system prompt names a transcript path, so eval.md:22 and automate-me/SKILL.md:29 sent agents to a source that does not exist. automate-me now resolves the session transcript tree through skill://omp-mechanics/references/session-store-on-omp.md, with a rule that reads the raw upstream sentence before the directory token rewrites it. The history://<id> rule now covers eval's "this path" as well as "the path", so eval reads the same as session-pickup and show-me-your-work.
Each defect below shipped while check-port printed PASS. Each check now fails on it, and test-gate-mutations.sh has a case that proves it. - rule replacement &: a bare & in a replacement splices the match back in (73f30d9 rule 512). - rule paren groups: an ERE group around literal parentheses that the replacement doubles (73f30d9 rules 538, 551, 560). - rule wildcards, swallow ledger: omp-port/swallow.tsv holds the sha256 of every raw upstream span each .* or .+ rule matches. A mid-line wildcard with no row, a missing row, a changed hash, or a row on a plain rule fails (4275556 rule 34; rules 35 and 89 at c742dd6). - rule staleness: fails an exemption on a rule that is not stale, and marks d, a, and addressed rules too, so rule 109 is now probed. - rule shadowing: fails a rule whose own raw match an earlier rule rewrites, unless omp-port/shadow-exempt.tsv records it (4275556 rule 228). Rules 156 and 170 are recorded. - owned drift: owned.txt records the upstream object last read for each owned path. setup-pstack still records the 519d36a tree, so this commit fails on ab1f702's unread edit. apply_patches runs patch --force, so a patch that no longer applies exits instead of waiting at a prompt on a terminal. apply_rules runs one sed per batch of files in parallel. Also: the readonly negation word boundary, the feature map grep, four retired-label terms and CHECK_PORT_DOCS_ROOT removed, a frontmatter name label, and the staleness, blank-exemption, patch-failure, and target-builds mutations restored. The harness runs its cases in parallel.
`git diff d533da1 800811d` shows one changed sentence in step 3(c). Arena's cross-judge pool used to pick one value whose family differs from the parent's when possible. Since ab1f702 Arena runs one judge, tries only values from another family in list order, and never judges with an `inherit-parent` or `auto` value. The owned setup-pstack has no cross-judge pool list to configure. On omp the judge is an exact discovered agent from another family, and the arena rule in rules.sed already says an agent that inherits the parent's model never judges. The owned copy needs nothing, so owned.txt now records the c742dd6 tree.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #33 (
sync/upstream-c742dd6, head bb0e701).Every defect in #33's two fix rounds shipped while
check-portprinted PASS. This PR makes each of those defect classes fail the gate, with a mutation case intest-gate-mutations.shthat proves the check fires. A maintainer syncing upstream now gets a red gate where they used to get a spliced sentence or a dropped one. Theupstream-syncworkflow runs the same checks against the candidate sha before it auto-merges.What fails now
rule replacement &&in a replacementopening-a-pr.mdthree timesrule paren groups(((rule wildcards.*with no$anchor and noswallow.tsvrow, or a row naming a plain rule(\b.*)?\.swallow ledgerrule stalenessd,aand addressed rules toorule shadowingshadow-exempt.tsvrecords itshadow exemptionsshadow-exempt.tsvrow naming no rule or twoowned driftowned.txtrecords as readrule livenessnow prints a cause count instead of 116 rule lines: 88 stale, 2 shadowed, 25 consumed nets, 1 catch-all.Other changes
apply_patchesrunspatch --force. Plainpatchwaits atFile to patch:orAssume -R?whensync-upstream.shprints the build to a terminal.--batchwas not an option, because it answersAssume -R?yes and reverts the hunk.apply_rulesruns onesed -E -iper batch of 64 files, in parallel.-iimplies-s, so range addresses still end at each file. The rebuild is byte-identical, andreproduciblepasses.feature mapand the label drops "pin", four retired-label terms that matched nothing are gone,CHECK_PORT_DOCS_ROOTis gone, the frontmatter check has a label, andlib.sh's dead pipe in the staleness probe is gone with the function it lived in.CHECK_PORT_SKIP_MUTATIONSbecameCHECK_PORT_NESTED, which also skips the worktree audit in nested runs.rules.sedtranslates the two false transcript claims.eval.md:22now says "(history://<id>is the documented surface)", like session-pickup.automate-me/SKILL.md:29resolves the transcript tree throughskill://omp-mechanics/references/session-store-on-omp.md.stale-exempt.tsvgains a row for rule 109 and a final newline. Without the newline, an appended mutation row edited the last row instead.ab1f702 and the owned setup-pstack
Commit fae861e records
owned.txtwith the 519d36a tree, d533da1, and its gate fails on exactly that,skills/setup-pstack: upstream moved at c742dd6, reviewed d533da1…, now 800811d….git diff d533da1 800811dshows one sentence changed in step 3(c). Arena's cross-judge pool used to pick one value whose family differs from the parent's when possible. Since ab1f702 Arena runs one judge, tries only other-family values in list order, and never judges withinherit-parentorauto. The owned skill configures no cross-judge pool list. On omp the judge is an exact discovered agent from another family, and the arena rule already says an agent that inherits the parent's model never judges. The owned copy needs nothing, so f381939 records 800811d.Recorded, not fixed
rules.sedwas open to this PR only for the transcript claims, so two shadowed rules are recorded inshadow-exempt.tsvinstead of fixed.agent mode (readonly: false)) sits behind rule 155'sreadonly: falsetoken rule. It has no effect at c742dd6, because the reflect sentence rules consume both upstream lines. Moving it above rule 155 makes it fire.subagent_type: "poteto-agent") duplicates rule 149's output. Deleting it changes nothing.Runtime
Measured on this 16-core machine, two runs each,
CANON=/tmp/cursor-plugins bash omp-port/check-port.sh. bb0e701 took 87.9 s and 87.7 s. f381939 took 37.4 s and 37.6 s. The worktree audit is about 20 s in both. The rest of the drop is the batchedsed: one build went from about 10 s to about 1.4 s. The harness went from 13.7 s with 17 cases to 9.9 s with 37. CI runners have fewer cores, so the parallel part will save less there (inferred, not measured).Verification
CANON=/tmp/cursor-plugins bash omp-port/check-port.shat f381939 endscheck-port: PASS, includinggate mutations PASS 37 mutations rejected, each by its own check.bash omp-port/test-gate-mutations.shat f381939 endsmutation tests: PASS, 37 cases rejected. New cases:bare & in a replacement,parentheses as a group,greedy mid-line wildcard,swallow row on a plain rule,swallowed text changed,wildcard rule with no row,exemption on a live rule,delete rule matching no line,rule behind its own duplicate,shadow row naming no rule,owned path moved upstream,patch prompt on a terminal,readonly after another,guide feature map,bare skill command,required item field. Restored:rule matching no upstream text,whitespace exemption row,patch that no longer applies,target that no longer builds.bash -npasses on everyomp-port/*.sh.CANON=/tmp/cursor-plugins bash omp-port/sync-upstream.sh rebuildchanged onlyautomate-me/SKILL.mdandpoteto-mode/playbooks/eval.md.rule_table,mark_rulesand span output under Ubuntu's mawk 1.3.4 20240123. The readonly check fails under that mawk, and bb0e701's regex misses the negation onwhy/SKILL.md:13there too, so that predates this PR. CI passes it, so CI'sawkis probably gawk (inferred, not checked).Each new check fails on the historical input, run in throwaway copies of this branch with
CHECK_PORT_NESTED=1 bash omp-port/check-port.sh.73f30d9's
rules.sed:4275556's
rules.sedandstale-exempt.tsv:swallow.tsvhashed at 519d36a and gated at c742dd6 fails on rules 25, 35, 89, 93, 140 and 141, which are the rules whose swallowed text upstream changed:apply_patcheswith the oldpatch ... <"$p"and the terminal deadline cut to 20 s:With
--batchinstead of--force:Replaces #38, which GitHub closed when its base branch was deleted on merging #33. Same commits rebased onto main; the tree is byte-identical to the reviewed head f381939.