feat: add direct QEMU, Goss and native Molecule actions by neilime · Pull Request #8 · hoverkraft-tech/ci-github-iac

neilime · 2026-10-31T11:22:00Z

Consumers can test infrastructure through focused actions while keeping environment policy in their own scenarios.

Add direct QEMU alongside Lima behind vm/create's backend selector, with explicit configuration, private SSH outputs, startup rollback and verified cleanup.
Add checksum-pinned Goss execution to vm/test, including native reports, summaries, PR comments and artifacts. Empty and fully skipped suites fail. Custom guest and runner commands remain supported.
Add ansible/test-molecule for native scenarios. It owns pinned runtime setup, bounded execution and report publication. The scenario owns inventory, playbooks, dependencies and sequence; no VM or reboot is required. Interrupted runs retry bounded native cleanup and remain failures.
Keep short GitHub Script logic inline. Remove the five entrypoint wrappers and the small Molecule report module; retain adjacent modules for complex validation, execution and VM operations.
Dedicate one hosted workflow to each public action: vm/create, vm/test, vm/destroy, ansible/test, and ansible/test-molecule. Keep backend variants inside each action workflow, and run all five in CI. Use test-action-* caller IDs and explicit Arrange/Act/Assert steps following ci-github-common; keep unconditional cleanup separate. Preserve native failures and reports; check rollback before fallback cleanup and verify repeated destruction and unmanaged-data preservation.
Organize 53 focused unit tests into seven native suite() blocks with test() cases and explicit Arrange, Act and Assert comments. Each action owns its dependency-free npm package and calls ci-github-nodejs/actions/test with command: test:ci; pretest:ci prepares the coverage directory and test:ci generates a gitignored LCOV report. Unit tests cover input boundaries, ownership, native report integrity and injected failures. Hosted scenarios cover ordinary execution, reporting and cleanup. Shared CI composes the action workflows; make test runs their suites locally and stops on failure.
Keep ansible/test for playbook and collection sanity tests. Workflows own the VMs they create and always schedule destruction. Linting stays separate.
Record backend choice, action responsibilities, native scenarios and script placement in one consolidated ADR, linked from the existing lifecycle decision.
Maintain the Goss release and Linux x86-64 archive checksum together through Renovate's release-attachment datasource, preserving the existing JSON format and runtime pin. Use Renovate's native .node-version manager to group updates across all five action packages.

The Proxmox pilot in digital-household #7 explicitly defines its SSH inventory, baseline application, idempotence and reboot sequence. Released playbook, collection and custom VM command inputs remain supported; the unreleased VM-specific Molecule interface is removed.

Validation: make test passes all 53 focused unit tests in seven suites, with explicit AAA sections in every test. All five action packages retain their dependency-free install and CI coverage scripts. Local lint and documentation links pass. Hosted VM tests also assert private report-directory permissions. Renovate strict configuration validation passes and its native Node version manager detects all five files. Its Goss extractor identifies the live upstream archive checksum, and its replacement logic updates the version and checksum together in a simulated release update. Shared hosted CI passes all 24 jobs at 72acbaa6ca9745cf0811a1a7a685dd12dcc2111b, including all five unit suites running test:ci from their action directories through the released Node.js test action. The Proxmox consumer run passed at c5482db6d1d790bea5d9b2578c89a766240a5912, using shared action revision a752dd6e1d27d80e510ae533ca71c2cb045cabb9; action runtime code is unchanged by the subsequent workflow and documentation changes. Hosted scenarios cover operation without a VM, deliberate failures, timeouts, Lima/QEMU, idempotence, reboot, native reporting and cleanup.

Intended for release 0.3.0 after review and shared/consumer qualification. Consumers must replace preview pins with the reviewed release commit before merge.

github-actions · 2026-10-31T11:22:09Z

Hi, thank you for creating your PR, we will check it out very soon

github-actions · 2026-10-31T11:22:32Z

Ansible tests - playbook - success

Test Results

Status	Count
✅ Passed	1
❌ Failed	0
⏭️ Skipped	0
Total	1

✅ Pass Rate: 100.0%

github-actions · 2026-10-31T11:22:39Z

Ansible tests - playbook - failure

Test Results

Status	Count
✅ Passed	0
❌ Failed	1
⏭️ Skipped	0
Total	1

❌ Pass Rate: 0.0%

Failed Tests

[localhost] Exercise a synthetic Ansible test assertion: Assert the selected fixture succeeds (suite: valid)
- File: tests/fixture/ansible/valid.yml:7
- Error: Task failed: Action failed: Assertion failed

github-actions · 2026-10-31T11:23:01Z

Ansible tests - sanity - failure

Test Results

Status	Count
✅ Passed	6
❌ Failed	1
⏭️ Skipped	27
Total	34

❌ Pass Rate: 17.6%

Failed Tests

yamllint (suite: ansible-test)
- File: sanity
- Error: The test `ansible-test sanity --test yamllint` failed with 2 errors:

github-actions · 2026-10-31T11:23:20Z

QEMU guest reports

Test Results

Status	Count
✅ Passed	1
❌ Failed	0
⏭️ Skipped	0
Total	1

✅ Pass Rate: 100.0%

Coverage

Metric	Covered	Total	Percentage
Lines	1	1	100.00%

Overall: 100.00% 🟢
🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩

github-actions · 2026-10-31T11:23:26Z

QEMU runner reports

Test Results

Status	Count
✅ Passed	0
❌ Failed	1
⏭️ Skipped	0
Total	1

❌ Pass Rate: 0.0%

Failed Tests

expected-failure (suite: Unknown Suite)
- Error: Synthetic failure

github-actions · 2026-10-31T11:23:32Z

Ansible tests - sanity - success

Test Results

Status	Count
✅ Passed	7
❌ Failed	0
⏭️ Skipped	27
Total	34

✅ Pass Rate: 20.6%

github-actions · 2026-10-31T11:23:44Z

Super-linter summary

Language	Validation result
BIOME_FORMAT	Pass ✅
BIOME_LINT	Pass ✅
CHECKOV	Pass ✅
GITHUB_ACTIONS	Pass ✅
GITHUB_ACTIONS_ZIZMOR	Pass ✅
GITLEAKS	Pass ✅
GIT_COMMITLINT	Pass ✅
GIT_MERGE_CONFLICT_MARKERS	Pass ✅
JSCPD	Pass ✅
MARKDOWN	Pass ✅
MARKDOWN_PRETTIER	Pass ✅
NATURAL_LANGUAGE	Pass ✅
PRE_COMMIT	Pass ✅
RENOVATE	Pass ✅
SPELL_CODESPELL	Pass ✅
TRIVY	Pass ✅
YAML	Pass ✅
YAML_PRETTIER	Pass ✅

All files and directories linted successfully

For more information, see the GitHub Actions workflow run

Powered by Super-linter
Super-linter revision: 2da136927bd4a73596db63044b504547c62cb854
Super-linter version: 9.0.0

github-actions · 2026-10-31T11:24:58Z

VM tests - success - first

Test Results

Status	Count
✅ Passed	1
❌ Failed	0
⏭️ Skipped	0
Total	1

✅ Pass Rate: 100.0%

github-actions · 2026-10-31T11:25:04Z

VM tests - success - repeated

Test Results

Status	Count
✅ Passed	1
❌ Failed	0
⏭️ Skipped	0
Total	1

✅ Pass Rate: 100.0%

Coverage

Metric	Covered	Total	Percentage
Lines	1	1	100.00%

Overall: 100.00% 🟢
🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩

github-actions · 2026-10-31T11:25:08Z

VM tests - assertion-failure - first

Test Results

Status	Count
✅ Passed	0
❌ Failed	1
⏭️ Skipped	0
Total	1

❌ Pass Rate: 0.0%

Failed Tests

deliberate-failure (suite: guest-fixture)
- File: ./tests/fixture/fail.sh
- Error: Synthetic assertion failure

github-actions · 2026-10-31T11:25:16Z

VM tests - assertion-failure - repeated

Test Results

Status	Count
✅ Passed	1
❌ Failed	0
⏭️ Skipped	0
Total	1

✅ Pass Rate: 100.0%

Coverage

Metric	Covered	Total	Percentage
Lines	1	1	100.00%

Overall: 100.00% 🟢
🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩

github-actions · 2026-10-31T14:04:45Z

Goss - __test-actions-qemu.yml - valid - 0

Test Results

Status	Count
✅ Passed	2
❌ Failed	0
⏭️ Skipped	0
Total	2

✅ Pass Rate: 100.0%

github-actions · 2026-10-31T14:04:53Z

Goss - __test-actions-qemu.yml - invalid - 0

Test Results

Status	Count
✅ Passed	0
❌ Failed	1
⏭️ Skipped	0
Total	1

❌ Pass Rate: 0.0%

Failed Tests

File /iac-missing-synthetic-file exists (suite: goss)
- Error: Test failed

github-actions · 2026-10-31T14:05:01Z

Goss - __test-actions-qemu.yml - empty - 0

Test Results

Status	Count
✅ Passed	0
❌ Failed	1
⏭️ Skipped	0
Total	1

❌ Pass Rate: 0.0%

Failed Tests

consumer-command (suite: disposable-vm)
- Error: Guest assertions or report collection failed; inspect test outputs

github-actions · 2026-10-31T14:05:20Z

Molecule - success

Test Results

Status	Count
✅ Passed	10
❌ Failed	0
⏭️ Skipped	0
Total	10

✅ Pass Rate: 100.0%

github-actions · 2026-10-31T14:05:36Z

Molecule - assertion-failure

Test Results

Status	Count
✅ Passed	3
❌ Failed	2
⏭️ Skipped	0
Total	5

❌ Pass Rate: 60.0%

Failed Tests

scenario (suite: molecule)
- Error: Molecule failed, timed out or produced no native assertions
[smoke] Verify the disposable test configuration: Assert the expected content (suite: verify)
- File: tests/fixture/molecule/smoke/verify.yml:10
- Error: Task failed: Action failed: Assertion failed

github-actions · 2026-10-31T14:05:53Z

Molecule - idempotence-failure

Test Results

Status	Count
✅ Passed	5
❌ Failed	1
⏭️ Skipped	0
Total	6

❌ Pass Rate: 83.3%

Failed Tests

scenario (suite: molecule)
- Error: Molecule failed, timed out or produced no native assertions

github-actions · 2026-10-31T14:06:13Z

Lima state - success

Test Results

Status	Count
✅ Passed	2
❌ Failed	0
⏭️ Skipped	0
Total	2

✅ Pass Rate: 100.0%

github-actions · 2026-10-31T14:06:34Z

Lima state - assertion-failure

Test Results

Status	Count
✅ Passed	2
❌ Failed	0
⏭️ Skipped	0
Total	2

✅ Pass Rate: 100.0%

…tests

github-actions · 2026-11-03T09:23:45Z

Molecule local scenario - success

Test Results

Status	Count
✅ Passed	7
❌ Failed	0
⏭️ Skipped	1
Total	8

✅ Pass Rate: 87.5%

github-actions · 2026-11-03T09:24:15Z

Molecule local scenario - failure

Test Results

Status	Count
✅ Passed	4
❌ Failed	2
⏭️ Skipped	1
Total	7

❌ Pass Rate: 57.1%

Failed Tests

[local-fixture] Verify the local test resource: Assert configuration and selected outcome (suite: assertions)
- File: tests/fixture/molecule/local/assertions.yml:14
- Error: Task failed: Action failed: Assertion failed
scenario (suite: molecule)
- Error: Molecule failed, timed out or produced no native assertions

github-actions · 2026-11-03T09:24:32Z

Molecule local scenario - timeout

Test Results

Status	Count
✅ Passed	3
❌ Failed	1
⏭️ Skipped	0
Total	4

❌ Pass Rate: 75.0%

Failed Tests

scenario (suite: molecule)
- Error: Molecule failed, timed out or produced no native assertions

github-actions · 2026-11-03T10:31:01Z

Goss - vm/test - qemu - valid

Test Results

Status	Count
✅ Passed	2
❌ Failed	0
⏭️ Skipped	0
Total	2

✅ Pass Rate: 100.0%

github-actions · 2026-11-03T10:31:09Z

Goss - vm/test - qemu - invalid

Test Results

Status	Count
✅ Passed	0
❌ Failed	1
⏭️ Skipped	0
Total	1

❌ Pass Rate: 0.0%

Failed Tests

File /iac-missing-synthetic-file exists (suite: goss)
- Error: Test failed

github-actions · 2026-11-03T10:31:18Z

Goss - vm/test - qemu - empty

Test Results

Status	Count
✅ Passed	0
❌ Failed	1
⏭️ Skipped	0
Total	1

❌ Pass Rate: 0.0%

Failed Tests

consumer-command (suite: disposable-vm)
- Error: Guest assertions or report collection failed; inspect test outputs

feat(vm): support direct QEMU and runner-side tests

0ffd578

neilime mentioned this pull request Oct 31, 2026

test(proxmox): validate installation through shared VM actions neilime/digital-household#7

Open

fix(vm): isolate runner script input from test tools

cb8070a

neilime marked this pull request as ready for review October 31, 2026 11:34

feat: standardize VM verification with Goss and Molecule

51d370f

fix: preserve VM login identity for Molecule reboot tests

4100e67

neilime changed the title ~~feat(vm): support direct QEMU and runner-side tests~~ feat(vm): add direct QEMU and opinionated Goss and Molecule tests Oct 31, 2026

neilime added 3 commits October 31, 2026 15:14

refactor: share post-reboot idempotence policy

e218764

fix: retain verification fixtures across guest reboot

59e4e1b

docs(vm): lead with the shared VM lifecycle

ca964eb

neilime added 3 commits November 2, 2026 17:53

refactor(actions): separate VM deployment tests from Ansible content …

aa88615

…tests

refactor(ansible): let Molecule scenarios own their environment

90e3df6

fix(ci): resolve the Molecule fixture path within steps

6261d1b

neilime changed the title ~~feat(vm): add direct QEMU and opinionated Goss and Molecule tests~~ feat: add direct QEMU, Goss and native Molecule actions Nov 3, 2026

neilime added 2 commits November 3, 2026 10:55

refactor: inline simple GitHub action scripts

a752dd6

test: dedicate hosted workflows to each public action

4d5fa69

neilime added 10 commits November 3, 2026 11:33

test: align fixture calls with workflow conventions

1b2a2f9

test: follow Arrange Act Assert workflow conventions

323d9dc

test: run unit checks in their owning action workflows

c07599b

ci: remove redundant aggregate check job

e4a6f4c

docs: consolidate infrastructure test action decisions

1a38504

ci: automate Goss release and checksum updates

eaac7f4

test: keep unit packages beside their actions

4d41bce

ci: maintain Node versions and add CI test scripts

2cb280a

test: focus unit coverage on distinct failure contracts

788f5f5

test: organize action cases into suites with AAA sections

72acbaa

Conversation

neilime commented Oct 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 31, 2026

Uh oh!

github-actions Bot commented Oct 31, 2026

Ansible tests - playbook - success

Test Results

Uh oh!

github-actions Bot commented Oct 31, 2026

Ansible tests - playbook - failure

Test Results

Uh oh!

github-actions Bot commented Oct 31, 2026

Ansible tests - sanity - failure

Test Results

Uh oh!

github-actions Bot commented Oct 31, 2026

QEMU guest reports

Test Results

Coverage

Uh oh!

github-actions Bot commented Oct 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

QEMU runner reports

Test Results

Uh oh!

github-actions Bot commented Oct 31, 2026

Ansible tests - sanity - success

Test Results

Uh oh!

github-actions Bot commented Oct 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Super-linter summary

Uh oh!

github-actions Bot commented Oct 31, 2026

VM tests - success - first

Test Results

Uh oh!

github-actions Bot commented Oct 31, 2026

VM tests - success - repeated

Test Results

Coverage

Uh oh!

github-actions Bot commented Oct 31, 2026

VM tests - assertion-failure - first

Test Results

Uh oh!

github-actions Bot commented Oct 31, 2026

VM tests - assertion-failure - repeated

Test Results

Coverage

Uh oh!

github-actions Bot commented Oct 31, 2026

Goss - __test-actions-qemu.yml - valid - 0

Test Results

Uh oh!

github-actions Bot commented Oct 31, 2026

Goss - __test-actions-qemu.yml - invalid - 0

Test Results

Uh oh!

github-actions Bot commented Oct 31, 2026

Goss - __test-actions-qemu.yml - empty - 0

Test Results

Uh oh!

github-actions Bot commented Oct 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Molecule - success

Test Results

Uh oh!

github-actions Bot commented Oct 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Molecule - assertion-failure

Test Results

Uh oh!

github-actions Bot commented Oct 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Molecule - idempotence-failure

Test Results

neilime commented Oct 31, 2026 •

edited

Loading

github-actions Bot commented Oct 31, 2026 •

edited

Loading

github-actions Bot commented Oct 31, 2026 •

edited

Loading

github-actions Bot commented Oct 31, 2026 •

edited

Loading

github-actions Bot commented Oct 31, 2026 •

edited

Loading

github-actions Bot commented Oct 31, 2026 •

edited

Loading