Skip to content

CI merge gate: pinned lint/secret/workflow checks and an 81-test regression suite - #2

Merged
jsdiazp merged 9 commits into
optimize/dotfilesfrom
ci/merge-gate
Nov 3, 2026
Merged

jsdiazp merged 9 commits into
optimize/dotfilesfrom
ci/merge-gate

Conversation

@jsdiazp

@jsdiazp jsdiazp commented Nov 3, 2026 •

Copy link
Copy Markdown
Owner

Summary

Adds a CI pipeline whose single required check, CI gate, passes only when every safety check passes. Stacked on #1 (the tests exercise its scripts); GitHub retargets this PR to main when #1 merges.

Pipeline (.github/workflows/ci.yml, replaces lint.yml)

  • lint: shellcheck 0.11.0 + shfmt 3.14.1 over setup, scripts, CI scripts and tests.
  • actionlint: lints the workflows (and their run: blocks with the pinned shellcheck).
  • secrets: gitleaks over the full history, --redact so nothing leaks into the public log.
  • tests: the new suite on Ubuntu (bash 5) and macOS (real /bin/bash 3.2.57).
  • gate: if: always(), fails unless all of the above succeeded, so a skipped or cancelled job can't slip through.
  • pull_request only (never pull_request_target), read-only token, no secrets, no path filters. actions/checkout pinned to a commit SHA; every other tool is a release binary verified against GitHub's published SHA-256 digest by .github/scripts/install-tool.sh. Dependabot keeps the SHA pin current.

Tests (tests/, 81 tests, no dependencies, about 30 s)
Each test runs in a scrubbed env -i process with a scratch HOME, stub binaries first on PATH, and pinned ASCII output, so nothing ambient can change a result and no real package manager runs. They pin the behaviors that were real bugs or destructive-action guards in #1: tasks run under bash, nested run_step, spinner never outliving its shell, --yes can't answer the Homebrew-uninstall prompt, ubuntu.sh runs nothing unless confirmed, .zshrc/.zprofile are never deleted, one summary row per updater, failing tasks exit non-zero, tmux/alacritty configs parse.

Found while writing the tests: sed_inplace aborted (BSD sed) or silently replaced the link (GNU sed) when ~/.zshrc is a symlink into a dotfiles repo, so setup_zsh broke for that common setup. Fixed in common.sh with a regression test.

How I checked the tests are real

For each guard I reverted it in a scratch copy and confirmed a test fails (zsh interpreter, strict confirm, ubuntu consent gate, .zprofile deletion, spinner parent check, summary double-recording, temp-file leaks, tmux/alacritty edits, and more). Only one mutant survived: removing a local that affects nothing but a step's elapsed time, which the suite deliberately doesn't assert.

After merging

main is unprotected today. Once #1 and this PR are merged and CI gate has run on main, a ruleset requiring a PR and the CI gate check (admin bypass allowed, 0 approvals, no force-push or deletion) will be created. It can't be created earlier without blocking #1 on a check that doesn't exist there yet.

Known gaps

  • Windows .ps1 scripts and vim/.vimrc (downloads plugins when run) are not executed by CI.
  • tmux 3.8 silently discards a config with a parse error; the config test therefore pins key options (prefix, mouse, vi keys) instead of relying on error output.
  • A step function that calls bare exit (not die) leaves one empty capture file in $TMPDIR; harmless litter, documented by a test.
  • The first Ubuntu run exposed a real hermeticity gap (the runner image ships composer, which an unstubbed test then ran for real). Tests now get a PATH of stubs plus a filtered farm of system tools with package managers, git/curl and sudo removed, so an unstubbed tool is "command not found" on any image. Both legs are green.

🤖 Generated with Claude Code

jsdiazp and others added 9 commits November 3, 2026 08:05
tests/run.sh runs each test_* function in a scrubbed env -i process with a
scratch HOME/TMPDIR and stub binaries first on PATH. tests/lib provides
assertions, stub/call-log helpers and a repo-copy helper; test_harness.sh
tests the framework itself. Works on macOS bash 3.2 and bash 5.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Replaces lint.yml with ci.yml: lint (shellcheck+shfmt), actionlint, gitleaks
secret scan over full history, and the test suite on ubuntu (bash 5) and
macOS (bash 3.2). The gate job fails unless every job succeeded, so the
ruleset only ever needs that one check. actions/checkout is pinned to a
commit SHA; every other tool is a checksum-verified release installed by
.github/scripts/install-tool.sh. Dependabot keeps the SHA pin current.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
BSD sed aborts on a symlinked file and GNU sed replaces the link with a
regular file, so setup_zsh broke (or silently detached) a ~/.zshrc that
points into a dotfiles repo. sed_inplace now resolves the link chain
(bounded, so a cycle is an error) and edits the real file.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Pins nested run_step (crashed before), one summary row per step, soft step
never dying, temp-file cleanup, strict confirm ignoring --yes, as_root,
safe_link/backup failure leaving the destination intact, append_once
idempotence, menu return codes, and sed_inplace editing through symlinks.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Tasks run under bash (word splitting), failing tasks exit non-zero without a
success banner, -y/--yes export, menu survives failures; update.sh brew
order, one row per ecosystem, failure isolation, skip path, Linux path.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
--yes never uninstalls Homebrew, ~/.zprofile/.zshrc are never deleted,
symlinked ~/.zshrc survives, brew gets separate args, starship only when
missing; ubuntu.sh runs nothing unless confirmed; docker/swap sequencing.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Spinner animates and cleans up under a real pty, never outlives its shell
(survivor check) and nested failures surface the error; alacritty.toml
parses and keeps the 'decorations = "Transparent"' literal that shell.sh
seds; tmux.conf loads with prefix C-s/mouse/vi keys; every vim plugin
fragment the .vimrc sources exists.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
The first Ubuntu CI run executed a real 'composer self-update' because the
runner image ships composer and one test did not stub it, and the
'missing tool' test had to skip itself for the same reason.

Tests now get PATH = stubs + a symlink farm of /bin and /usr/bin minus
package managers, network/VCS tools and privileged commands, so an
unstubbed tool is 'command not found' and absence is deterministic on any
image. The OMZ test now stubs git explicitly instead of relying on the
host's.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
@jsdiazp jsdiazp changed the title CI merge gate: pinned lint/secret/workflow checks and a 78-test regression suite CI merge gate: pinned lint/secret/workflow checks and an 81-test regression suite Nov 3, 2026
@jsdiazp
jsdiazp added this pull request to stack #3 November 3, 2026 13:51
@jsdiazp
jsdiazp merged commit bffc6cb into main Nov 3, 2026
6 checks passed
@jsdiazp
jsdiazp deleted the ci/merge-gate branch November 3, 2026 14:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant