Repository navigation
ci: release-grade install matrix, upgrade test and docs-as-tests - #67
Merged
Merged
Conversation
Both flags are parsed and documented in docs/rule-packs.md but were missing from `penthera --help`. Found by the new docs-commands check. AI-Assisted: Claude Code with Claude Opus 5.5; found the gap with the docs-commands check and added the two help lines Co-Authored-By: Claude Opus 5.5 <[email protected]>
…rom 1.7.0, and every documented command - install-matrix.yml: npm pack once; install the tarball into an empty project on ubuntu/macos/windows x Node 22/24 and run npx penthera (--version, --help, init --yes --dry-run, deps, deps --base, agents, --repo, hook install + a real git commit through the hook, ui --demo answering 401, rules verify on the bundled packs). - Upgrade job: [email protected] from npm writes memory and ui state, then the tarball must verify and extend the same chain. - scripts/docs-commands.mjs: extracts every command in README, docs and SKILL.md, parse-checks subcommands and flags against --help and runs the offline-safe ones on a fixture. Runs in npm test and in the matrix against the installed package. - scripts/ci/net-guard.cjs: every Penthera process in these checks fails on a connection or DNS query past loopback. AI-Assisted: Claude Code with Claude Opus 5.5; designed and wrote the workflow, smoke runner, network guard and docs checker from the maintainer's release-verification brief Co-Authored-By: Claude Opus 5.5 <[email protected]>
AI-Assisted: Claude Code with Claude Opus 5.5; wrote the notes for the new install matrix, upgrade job, docs-commands check and the Windows symlink Co-Authored-By: Claude Opus 5.5 <[email protected]>
…ject On Windows, `git rev-parse --show-toplevel` prints D:/a/app with forward slashes. The wizard compared resolve()d report paths (D:\a\app\reports) against that root, so every write failed with "Refusing to write outside the project" and `npx penthera` / `penthera init` exited 1 in any git repo. Both git-root helpers now return native paths. Found by the Windows legs of the new install matrix. AI-Assisted: Claude Code with Claude Opus 5.5; traced the install-matrix failure to the git path format and normalized both helpers Co-Authored-By: Claude Opus 5.5 <[email protected]>
CodeQL (js/incomplete-sanitization) on the new report writers. AI-Assisted: Claude Code with Claude Opus 5.5; fixed the CodeQL alerts Co-Authored-By: Claude Opus 5.5 <[email protected]>
AI-Assisted: Claude Code with Claude Opus 5.5; added the changelog line Co-Authored-By: Claude Opus 5.5 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release-grade verification for 1.8.0. Until now CI ran Penthera only from the repo, on Linux. This PR tests the packed tarball the way users install it, on Linux, macOS and Windows. It also tests an upgrade from 1.7.0 and every command shown in the docs.
Found and fixed along the way (separate commits):
npx penthera/penthera initwas broken in any git repo. It exited 1 withRefusing to write outside the project: D:\…\reports\…. The cause:git rev-parse --show-toplevelprintsD:/a/appwith forward slashes, and that was compared againstresolve()d backslash paths. Both git-root helpers (src/cli/detect.js,lib/hook/index.js) now return native paths (6cf41d6).penthera --helpwas missing--rules-signersand--allow-unsigned-rules. Both are parsed and documented indocs/rule-packs.md(3399fc1).No version bump, no tag.
What's new
.github/workflows/install-matrix.ymlruns on PRs to main, pushes to main and workflow_dispatch. It usespermissions: contents: readand SHA-pinned actions (download-artifact v8.0.1 is the only new pin).npm packruns once (--ignore-scripts, since ci.yml already runs the tests), and the job checks the tarball's file list: required files present, no repo-only folders. The tarball is passed on as an artifact.npm install <tgz> --ignore-scripts), andscripts/ci/smoke-installed.mjsrunsnpx pentheraagainst a committed git copy oftests/fixtures/init/app(lockfile +.mcp.json+CLAUDE.md). Each check asserts the exit code and key output lines.[email protected]is installed from npm in a temp HOME and runs a--memoryscan plusui --demo. Then the tarball is installed over it, and the job confirms the lockfile now resolves tofile:. Old memory must verify (penthera memory verify),memory showmust read it, and a new--memoryscan must extend the same chain (6 → 9 events).ui --demomust start against the old~/.penthera.scripts/docs-commands.mjs+tests/docs-commands.test.js(runs innpm test, and in the matrix against the installed package on all three OSes).scripts/ci/net-guard.cjsis preloaded viaNODE_OPTIONSinto every Penthera process in these checks. A TCP/TLS connection or DNS query past loopback fails like a real network error and is logged, and the check asserts the log is empty. So a command that quietly falls back after a failed download still fails.Matrix results (run 39414300920)
Fresh-install checks:
--version;--help;init --yes --dry-run(detects git, lockfile and agent config, plans the hook and workflow, and writes nothing, which is checked with a file snapshot);deps;deps --base HEAD(spawns git);agents;--repo .. Next,hook install, then a realgit committhrough the hook: a staged key is blocked and a clean commit passes. That is Git for Windows' sh running node.exe by absolute path. Thenhook uninstall.ui --demois started in the background and polled until/answers 401 without a token, then stopped, and the check confirms the port is closed (no orphaned grandchild under npx).rules verifyruns on both bundled packs from the installed package, thenrules list. Last comes the zero-network assertion.Windows notes
initbug above. Everything else, including the hook's absolutenode.exepath, worked.autocrlf), and the smoke log prints it. deps, agents, init and the hook all read CRLF lockfiles and agent config correctly.npxgoes throughnpx.cmdwith a shell. git is spawned by init,deps --base, the hook and the commit. The ui server is stopped withtaskkill /T..cursor/skills/penthera→skills/pentherais a link only for people working on Penthera in Cursor. GitHub's Windows runner checks it out as a real link, but a default Git for Windows install gives a text file. This is documented in CONTRIBUTING ("Windows": Developer Mode +git clone -c core.symlinks=true). The npm package does not ship.cursor/, so users are unaffected. A Windows step records what the checkout got.Docs-command coverage
334 commands in 40 files (fenced bash blocks,
run:lines in yaml blocks, inlinepenthera …spans). Every Penthera command is parse-checked: the subcommand, the action (memory show,rules verify, …) and every flag must appear in the matching--help.--base origin/main→HEAD,--repo ./my-app→.). Commands run per file in document order, so--cra-init cra.jsonmakes--cra cra.jsonrunnable. All pass<url>,$URL, interactive (barepenthera,initwithout--yes,agent --loop), long-runningui,localhost:3000needing an app, inputs the fixture lacks (./my-pack,../base-checkout),--machine,--sign-keypenthera install(npm registry, 17 incl. inline),--suppliersnpx skills add,curl … | bash,/plugin,ssh-keygen,cp,jq, competitor CLIs indocs/compare/, …penthera ship/watchindocs/plan.md: planning docs (plan.md,vision-2.0.md,release-plan.md,research/,funding/) are reported, not failedThe only wrong command found was the help gap above. No README or doc text changed.
node scripts/docs-commands.mjs --report out.mdprints the full breakdown, and the matrix writes it to each job summary.Network
No new job hits the public internet from Penthera. npm does contact the registry in two places: installing the tarball fetches its one runtime dependency (
yaml), as any user install does, and the upgrade job installs[email protected]. Both are documented in the workflow header and CONTRIBUTING.Checks
npm testgreen locally (910 passed, docs test adds about 10 s) and in CI on Node 22/24.npm run audit:internalpasses all gates locally (REPORT.md not committed).Follow-ups (not in this PR)
FROM_VERSIONin install-matrix.yml to1.8.0.npm teston Windows/macOS is a separate, bigger change: some tests use/dev/nulland POSIX shells.🤖 Generated with Claude Code