Repository navigation
Prepare stable signed regression runners for durable guest authorization - #108
Merged
Merged
Conversation
This was referenced Oct 2, 2026
Open
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stabilize the regression runner before further capture-based tests
Verified for the targeted unattended-runner model: changed-source execution and two final-source headless boots passed without additional human interaction, grants or resets. Merged into
pocketatb8b70228d82bd0e8b208c2e34b4c94f6670fd9b2; no release was performed. Related to #18 and the foundation delivered by #101. This changes only the regression harness, its contracts, and documentation. The delivered change does not alter application behavior or replace the installed Notch app; no release is included.7d357dd30eb7fccd3d033dd862cead8e091af254and final sourceabfc8a2c7e986e9670ac7e23b33abd33a236df1dcompiled, packaged and verified. Both role executable/CDHashes differ; signer configuration, entitlements, identifiers, flags and normal designated requirements match. P1 passed admission plus the full suite; final P2 passed admission plus the full suite on two fresh boots, with actual recording access true throughout.Concrete startup correction
Native diagnostics exposed a real signing-input defect: Xcode's UI-test target fell back to the SDK entitlement file and claimed
com.apple.application-identifierwithout a provisioning profile. The guest'staskgated-helperrejected it before any test completed.Stable mode now supplies an explicit, empty, source-pinned
StableRunner.entitlementsusing ordinaryCODE_SIGN_ENTITLEMENTS. This replaces the generic SDK input, not XCTest's product-type or base/debug entitlements. Native comparison proved that only the unsupported application-ID claim disappeared; every other entitlement and the existing signing identity remained unchanged. ClearingDEVELOPMENT_TEAMwas tried, did not fix the claim, and was not retained.Verification now rejects that unprovisioned claim and requires actual Boolean sandbox and
get-task-allowentitlements. Base-entitlement injection remains enabled. The signing correction does not edit privacy databases, container access-control records or existing signed binaries.Native receipts additionally observe
CGPreflightScreenCaptureAccess()before interaction and, in B2, after restoration. These are non-prompting observations, not substitutes for output assertions or permission grants.Unattended update model
build_runner.py comparenow verifies two complete portable packages using independently supplied pins and each package's own source snapshot. It rejects signer, role, architecture, designated-requirement, entitlement or flag drift. It reports source/executable/code-hash changes without claiming consent or native acceptance. Unchanged packages are compatible, not cross-source proof.The setup, test-authoring and independent-suite workflows now distinguish:
New OS capabilities or intentional identity changes remain separate setup work. Routine source changes must not reactivate setup or make consent a per-run step.
Protected artifact lifecycle
Metadata files reappeared across reboot after setup browsing, while every approved file remained unchanged. Repeatedly restoring a writable package did not establish an unattended baseline.
Stable builds now protect Products and its verified runner-ancestor directories, deepest first, after signing verification. Signed app/test-bundle contents and modes are not changed. Only those declared container-mode transitions are allowed; newly appearing files fail rather than being incorporated into expectations.
Schema v2 binds the root mode and rechecks read-only, owned containers and native file-security state before/after signature verification. ACL lookup uses checked
fstatx_np/filesec, requiring populated owner/group/mode before accepting ACL absence. This closes the review finding where listing inference or libc's masked allocation failure could appear ACL-free.Legacy schema v1 remains inspectable for migration but cannot satisfy
verify --require-protected-products; comparison rejects a v2-to-v1 protection downgrade. Ad-hoc/unsigned defaults remain v1. Protection addresses incidental directory-entry writes in the named containers, not privileged writers or an owner changing permissions.Actual same-UID write attempts were denied with EACCES on the host and guest, including after archive extraction and both final-source boots. The fixed runtime path remained unchanged. No metadata appeared in the protected containers during the completed proof.
Validation and independent review
At current source
abfc8a2c7e986e9670ac7e23b33abd33a236df1d:Runtime evidence and completed acceptance
The corrected A2 runner exposed a second, distinct prerequisite: its stable identity was not yet authorized for the existing sandbox data container created by the old ad-hoc runner. An independent full-suite attempt timed out on the first About case; eight cases were unrun. A native process sample and container-service logs identified the startup wait. No completed test, capture-permission result or suite pass was claimed.
The human subsequently consented through the actual runner prompt; macOS recorded that consent for the expected Developer ID identity and existing test container. No additional programmatic approval occurred. A consent-only probe that timed out before the human response remains preserved as BLOCKED.
Fresh A2 headless verification passed 9/9 expected outcomes: four functional PASS results and five controls producing the registered raw two FAIL / three BLOCKED outcomes. Nine actual native tests, zero skips, eight inspected public Settings captures, complete UI/app/process/job/lock cleanup, and unchanged candidate/signatures/preferences/data. Both VMs stopped normally.
At that historical point,
screenCapturePreflightAccesswas false in all nine receipts. Native logs identified the existing Screen Capture entry's old ad-hoc cdhash requirement. Container consent fixed startup; a successful XCTest screenshot did not establish Screen Recording permission.The UI remove/re-add did not replace the effective legacy requirement. After the owner's explicit request, one targeted supported
tccutil reset ScreenCaptureoperation for this runner bundle was executed; native Reset/Delete events confirmed removal. It did not grant access and was not added to routine automation. A fresh boot confirmed the obsolete requirement was gone and the decision was Unknown/None.The human then enabled the exact stable runner again. Initial verification stopped at recurring
.DS_Storeinventory drift before it could measure permission; that was not another denied grant. Protected containers resolved this separate storage problem without further permission changes.The entire unprotected package is preserved. Protected P1 then passed one admission plus all nine expected suite outcomes: ten actual true capture preflights, zero skips and complete cleanup. The parent verified P1's complete protected inventory again after another cold boot before promoting P2.
Final P2 update/restart proof passed:
An extra read-only cross-reboot diagnostic initially compared kernel
st_devnumbers. Its failure is preserved: all 141 entries differed only in the mount-assigned device number, while persistent metadata and canonical inventory/signature checks remained unchanged. No test expectation, artifact or native run was repaired or retried.Local acceptance receipt:
evidence/pr108-unattended-runner-acceptance.jsonin the ignored VM environment, linked to the independently produced P1 and combined P2 reports. Raw images, credentials and VM data are not published.This proves the targeted model for the configured guest and approved capability set: normal rebuilt-runner updates and these suite runs needed no intervention. It does not claim all application features, new OS capabilities, future OS/signing changes, or indefinite/overnight reliability. Broader foundation work remains separate.
Earlier provisioning failures, incomplete result bundles, whole-package metadata-drift recovery, and failed administrative restoration remain historical evidence. Stale locks were archived only after exact ownership/liveness checks. No old failure has been relabeled as a pass.