Repository navigation
Conversation
Prerequisite for the arm64/macOS runtime port that follows, and useful on its own. A single zig process reached 77.5 GB on a 96 GB Mac and forced two physical resets (JetsamEvent-2026-09-20-170707: largestProcess "zig", lifetimeMax 77521 MB, its parent ruby only 100 MB -- one process, not the fork pool). Darwin does not enforce RLIMIT_AS, so the cap cannot be delegated to the kernel: `ulimit -v` is rejected outright and a test process allocated 10 GB straight through it. tools/mem-guard polls a command's process tree and kills it past a cap, descendants included so link subprocesses cannot outlive it; tools/zig-guard resolves the real zig and delegates. Every zig spawn in `clear` goes through the single ZIG constant, so routing that one value through the guard caps them all -- argv arrays and interpolated command strings alike -- rather than patching each call site. Default cap is a quarter of RAM; CLEAR_ZIG_MAX_RSS_MB tunes it and 0 disables. A kill is worded so it reads as a memory runaway, not a CLEAR compile error. Gemfile.lock gains arm64-darwin: the lock listed only x86_64-linux-gnu, so `bundle exec` could not resolve on a Mac at all and every Ruby tool died before it started. Co-Authored-By: Claude Opus 5 <[email protected]>
switch.S and onRoot.S each carry an `#if defined(__aarch64__)` block, but neither had ever been assembled: on aarch64 `#` introduces an immediate rather than a comment, so every `# ...` comment in those blocks is a syntax error, and `.section .text` is ELF-only syntax that Mach-O rejects. The blocks were dead code that looked like arm64 support. Building CLEAR for aarch64-linux fails on master at `stp x29, x30, [sp, #-48]! # Allocate ...`. - Comments inside the aarch64 blocks, and in switch.S's shared data header, become `//`, which both assemblers accept. The x86-64 bodies are untouched. - `.section .text` becomes `.text` in the aarch64 blocks. - Globals that Zig links against go through a new SYM() macro, because Mach-O prefixes C symbols with an underscore and ELF does not. - The ~500 lines of unguarded x86-64 in switch.S -- the SNAPSHOT/CRUMB macros, abi_panic, __morestack, __lessstack and the stack-stitch test harness -- are guarded so a non-x86 target no longer tries to assemble them. Only switch_context_asm and call_on_stack_asm are needed to build a CLEAR program; everything else in there is referenced solely by the x86 asm tests. - The `.weak` group is ELF-guarded: `.weak` is an ELF directive, and test_parent_ctx is declared weak but never defined (the label below it reads `test_paretn_ctx`), which only resolves because ELF lets a weak undefined symbol be 0. The shipping target is provably unchanged: assembled for x86_64-linux before and after, the disassembled text is identical (399 and 28 instruction lines) and the symbol tables are identical. The objects differ only in the source path embedded in debug info. aarch64-linux now builds a complete CLEAR executable (5.8 MB ELF, 676 MB peak). arm64-macos assembles both files but does not link yet: runtime/scheduler.zig is built on io_uring and eventfd, so std.os.linux.IoUring fails to compile on Darwin. Full macOS support needs a kqueue backend behind the existing RingType seam -- about twelve ring operations -- which is a separate subsystem port. Co-Authored-By: Claude Opus 5 <[email protected]>
…iber switch CLEAR now builds and runs natively on arm64 macOS. Three defects stood between here and there, and the last two were live bugs in the aarch64 path that no target had ever exercised. 1. The scheduler was built directly on io_uring, so std.os.linux.IoUring failed to compile on Darwin. Everything Linux-specific it touched -- IoUring, io_uring_cqe, kernel_timespec, POLL.IN, IORING_POLL_ADD_MULTI and eventfd -- is now named once in runtime/io-backend.zig and selected at comptime. On Linux every alias IS the std type, so Linux codegen is unchanged; elsewhere a poll()-based PollRing parks on a wake pipe (Darwin has no eventfd) and reports AsyncIoUnsupported for socket submissions. It is deliberately not an async backend: it supports what a compute-only program needs. A kqueue backend would drop in behind this same interface without touching the scheduler. 2. The aarch64 switch_context_asm wrote the wrong offsets. fiber-core.zig's Context packs x19 at 0x08, but the assembly stored it at 0x10 -- every register was off by eight, and fp/lr at 0x58/0x60 were never touched at all. 3. Worse, x29/x30 were neither saved nor restored, and a fiber's entry point was only written to the stack. That is the x86 model, where `ret` pops the return address; aarch64's `ret` branches to the link register. So the first switch into a fiber returned to whatever x30 happened to hold -- in a Debug build, the 0xCC stack fill, which is exactly the `Segmentation fault at address 0xcccccccccccccccc` this produced. The assembly now saves and restores x29/x30, and a fiber seeds .lr with its entry point on aarch64. Verified: hello world builds and prints on arm64-macos; x86_64-linux and aarch64-linux both still link complete executables; and switch.S assembled for x86_64-linux is still instruction-for-instruction identical to master. Known limitation: on macOS the build needs the LLVM backend (--safe or --optimized). The default self-hosted backend grows without bound there -- past 31 GB on hello world -- which is the runaway that originally took this machine down twice and is why tools/mem-guard exists. Co-Authored-By: Claude Opus 5 <[email protected]>
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #6 +/- ##
==========================================
- Coverage 92.01% 91.85% -0.17%
==========================================
Files 620 618 -2
Lines 262183 261160 -1023
Branches 20862 20862
==========================================
- Hits 241255 239876 -1379
- Misses 17558 17914 +356
Partials 3370 3370
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
|
Bencher Report| Project | CLEAR |
| Branch | mac-arm-support |
| Testbed | ubuntu-latest |
⚠️ WARNING: No Threshold found!Without a Threshold, no Alerts will ever be generated.
Click here to create a new Threshold
For more information, see the Threshold documentation.
To only post results if a Threshold exists, set the--ci-only-thresholdsflag.
Click to view all benchmark results
| Benchmark | leak-build-ms | Measure (units) x 1e3 | leak-count | Measure (units) | leak-run-ms | Measure (units) |
|---|---|---|---|---|---|---|
| benchmarks/concurrent/01_socket_throughput/bench | 📈 view plot | 6.56 units x 1e3 | 📈 view plot | 0.00 units | 📈 view plot | 7.53 units |
| benchmarks/concurrent/06_dynamic_spawn/bench | 📈 view plot | 6.32 units x 1e3 | 📈 view plot | 0.00 units | 📈 view plot | 42.39 units |
| benchmarks/concurrent/11_parallel_aggregation/bench | 📈 view plot | 6.43 units x 1e3 | 📈 view plot | 0.00 units | 📈 view plot | 16.24 units |
| benchmarks/concurrent/18_atomic_counter/bench | 📈 view plot | 6.34 units x 1e3 | 📈 view plot | 0.00 units | 📈 view plot | 5.76 units |
| benchmarks/inter-clear/04_concurrent_mvcc_fat_struct/bench | 📈 view plot | 6.58 units x 1e3 | 📈 view plot | 0.00 units | 📈 view plot | 166.06 units |
| benchmarks/sequential/03_alloc_throughput/bench | 📈 view plot | 6.07 units x 1e3 | 📈 view plot | 0.00 units | 📈 view plot | 7.54 units |
| benchmarks/sequential/08_sort/bench | 📈 view plot | 6.25 units x 1e3 | 📈 view plot | 0.00 units | 📈 view plot | 5.17 units |
| benchmarks/sequential/13_soa_layout/bench | 📈 view plot | 6.16 units x 1e3 | 📈 view plot | 0.00 units | 📈 view plot | 740.34 units |
| benchmarks/server/03_pathological/server | 📈 view plot | 6.43 units x 1e3 | 📈 view plot | 0.00 units | 📈 view plot | 1,002.55 units |
Tests for everything this branch added that can hold logic: io-backend.zig
(16 tests) and the two guard scripts (32 tests, 100% line coverage measured by
tracing, all 15 conditionals driven both ways).
Writing them found two real bugs in tools/zig-guard, both in the PATH fallback
that runs whenever CLEAR_REAL_ZIG is not set:
- `command -v -a` is a bashism. POSIX sh rejects it outright ("command: -a:
invalid option"), so the fallback found nothing and the guard exited 127
every time it was invoked outside ./clear -- exactly the way the handoff doc
tells people to use it. PATH is now walked by hand.
- The candidate test was `-x` alone, but a DIRECTORY carries the execute bit,
and an empty PATH element means the current directory. Run from the repo
root, whose `zig/` is a directory, the guard selected that directory as the
compiler. It now requires `-f` as well.
The io-backend tests pin the behaviour the scheduler depends on: the comptime
platform selection, the substituted Cqe/Sqe/KernelTimespec fields, the wake
channel (both the two-fd pipe shape and the one-fd eventfd shape), the
sec/nsec conversion and its non-positive clamp, the wait_nr == 0 fast path,
parking on a readable fd, the 50ms ceiling, the 1ms floor that keeps a
sub-millisecond timeout from busy-spinning, the no-wake-fd sleep, and every
async submission reporting AsyncIoUnsupported. Descriptor exhaustion covers
the pipe() failure path.
The guard suite drives its own awkward branches rather than declaring them
unreachable: `sysctl` is shadowed to fake a 2 GB host so the 4096 MB floor
fires, and `ps` is shadowed to fail so the degrade-to-zero path runs. SIGTERM
is delivered to prove the trap reaps the tree instead of orphaning a
multi-gigabyte zig.
compiler/spec/mem_guard_spec.rb runs the shell suite from the Ruby suite and
asserts the 100% figure, so a regression fails CI instead of waiting to be
noticed the next time zig tries to take a machine down.
Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01KSuF9ba5qccHAU1gZc9oQ2
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
There was a problem hiding this comment.
Decomplex found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.
Ran giga's analysis layers over the Zig and Ruby this branch adds. Findings on pre-existing code are left alone; these are the ones landing on new lines. espalier -- "no observed state writes: WakeFd#close". A real defect: close() closed the descriptors but left the numbers in the struct, so a second close would close whatever unrelated file had since inherited them. It now returns early when already closed and invalidates both fields. decomplex -- two convergence findings (>=2 independent detectors), both Function LCOM: - WakeFd.open held two disjoint data-flow components, the platform choice and the pipe/fcntl setup. openPipe() now owns the second. - copy_cqes held the timeout bookkeeping and the parking machinery, 17 locals across 10 statements. parkMillis() (pure, and now directly unit-tested at its boundaries) and park() split them. Also gave the two mutable fields one owner each: sqeSlot() for scratch, takePendingTimeout() for the take-and-clear. Convergence 2 -> 0, LCOM 2 -> 0, state heatmap 2 -> 1. Two findings stand as false positives, documented in the code so the next reader does not "fix" them: - scratch reads as eliminable intra-method state, but sqeSlot returns a POINTER that outlives the call; a local would dangle. - takePendingTimeout reads as derived-state staleness, but the local is deliberately the value from before the clear. zig-mutants -- 28 mutants, 19 killed / 9 survived (67.86%). The survivors were tests asserting self-consistently rather than against concrete values: a flipped have_io_uring still passed because the std type and the substitute both carry user_data/res, and "sqe.len == POLL_ADD_MULTI" compares a value to itself. Tests now pin the concrete type identity and literal constants, verify close() actually closes and is idempotent, cover fd 0 as a VALID wake fd (the emptiness test is "< 0", not "<= 0", because descriptor 0 is stdin), and exercise parkMillis's floor and ceiling directly. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01KSuF9ba5qccHAU1gZc9oQ2
There was a problem hiding this comment.
Espalier found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.
A flipped "write_fd != read_fd" survived mutation because the close test only checked the READ end. The mutant leaks the write end of every pipe and no test noticed. Both ends are now asserted closed.
Writing to the write end of a pipe whose read end is closed fails with EPIPE whether or not that descriptor was closed, so the previous assertion could not tell a leaked write end from a closed one -- and the flipped "write_fd != read_fd" mutant, which leaks the write end of every pipe, survived because of it. fcntl(F_GETFD) asks about the descriptor itself.
Both emptiness tests are "< 0", never "<= 0", because descriptor 0 is valid. With "<= 0" close() skips a WakeFd sitting on fd 0 and leaks it, and both mutants survived because no test put the channel there. stdin is redirected to a pipe and restored so fd 0 really is the channel.
park()'s emptiness test is "< 0" for the same reason close()'s is. With "<= 0" a wake fd of 0 falls to the usleep path and burns the whole interval instead of returning when the channel is readable, which is observable as elapsed time. Verified stable across repeated runs.
zig-mutants leaves three survivors on this file and none is a test gap. Two are provably equivalent, verified rather than assumed: Linux POLL.IN is 1, the same as the Darwin literal, so mutating that selection changes nothing; and "secs <= 0" vs "secs < 0" differ only at zero, where the else arm also yields 0. The third is the eventfd call inside the Linux-only branch, which does not execute on a Mac and is killable on Linux CI. Recording this in the source so the next person to run mutation testing does not re-derive it.
switch.S and onRoot.S annotate every x86-64 body with .cfi_startproc / .cfi_endproc; the aarch64 bodies had none, because they had never been assembled. Hand-written assembly with no unwind information is a hole in any stack walk that crosses it, and Zig's DebugAllocator walks the stack on every allocation. This is correctness hygiene, NOT a fix for the macOS unwinder hang documented alongside it: the hang reproduces identically with these directives in place, with the same stack. Claiming otherwise would be wrong. x86-64 output is unchanged -- switch.S and onRoot.S assembled for x86_64-linux are still instruction-for-instruction identical to master. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01KSuF9ba5qccHAU1gZc9oQ2
A hard hang (Mach-O unwinder faulting during DebugAllocator stack capture on a fiber stack, then the segfault handler re-entering it) and zig translate-c spinning at 100% CPU under parallel specs. Both make the Ruby suite unrunnable on macOS without exclusions, and both cost hours to find; the stack trace and the A/B that rules out CFI as the fix are written down so nobody repeats it.
All four reproduce on origin/master and are macOS-only, each an assumption that happens to hold on Linux. fmt_verifier_spec -- File.expand_path resolves a relative path against Dir.pwd, and Dir.pwd reports the SYMLINK-RESOLVED cwd. Dir.mktmpdir hands back /var/folders/... while /var is a symlink to /private/var, so the derived source_dir is the realpath and the spec compared against the unresolved path. The production derivation is correct; the expectation now says realpath, which is identical on Linux. pprof_converter -- resolve_addrs shelled out to addr2line unconditionally and died with ENOENT where binutils is absent (macOS ships atos). A missing OPTIONAL resolver should degrade, not abort: the loop already yields '?' for every field on empty output, so an unresolved profile is still a profile. It now tries addr2line, llvm-addr2line, gaddr2line and eu-addr2line, taking the first that runs. Spawning and rescuing beats probing PATH first -- one code path instead of two, and it cannot disagree with what exec would find. Probing PATH also broke the spec that stubs IO.popen. hostile_frontend -- Process.spawn(rlimit_as:) raises EINVAL on Darwin, which accepts RLIMIT_AS as a constant but rejects setrlimit for it; that is the same reason `ulimit -v` is rejected there. It now learns this by trying once rather than hardcoding a platform list. Losing the memory ceiling still leaves the CPU-time limit and the wall timeout bounding a hostile input; losing the spawn leaves nothing. Suite goes 7874 examples / 4 failures -> 7891 / 0. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01KSuF9ba5qccHAU1gZc9oQ2
The three c-ffi specs hung on macOS -- four zig translate-c processes at 100% CPU with a flat 32 MB RSS for 72 minutes. It is not the input and not the directory: the identical command on the identical fixture in the identical tmpdir finishes in 0.087s from a shell. Measured from Ruby on the same inputs: system(..., out: File::NULL) 0.06s system(..., out: <file>) 0.06s, 17675 bytes of correct output IO.popen(...) (stdout = pipe) never returns, 99% CPU So it is the pipe, and nothing else. compiler_zig_translate_c now redirects stdout and stderr to files in a tempdir and reads them back. Process.spawn rather than system, so a missing zig still raises Errno::ENOENT and the caller's rescue can turn it into "Zig is required to import C headers". The three specs go from hanging indefinitely to 16 examples, 0 failures in 3.09 seconds. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01KSuF9ba5qccHAU1gZc9oQ2
It hung forever on macOS with no output at all. Two causes, both fixed here so the underlying problem is at least VISIBLE: - zig's output was captured through a pipe (Open3.capture2e), and macOS zig wedges on a pipe -- the same defect that made zig translate-c spin. Captured through a file, the run finally says what happened: 'Segmentation fault at address 0x8' on the first of 148 generated cases. - a zig test binary that crashes can then sit in its own signal handler at 0% CPU indefinitely, so the wait is now bounded and reports the captured output. This does NOT fix that segfault, which is a real crash in generated semantic-mutant code on arm64 macOS. Ordinary fiber, stream and nested-BG programs all pass there, so it is narrow rather than 'fibers are broken'. Co-Authored-By: Claude Opus 5 <[email protected]> Claude-Session: https://claude.ai/code/session_01KSuF9ba5qccHAU1gZc9oQ2
`zig test` on the generated semantic-mutant suite crashed on its first case
with "Segmentation fault at address 0x8" and then wedged at 0% CPU in its own
signal handler. The fault address names the bug: aarch64 stack walkers follow
the frame-pointer chain, reading the saved FP at [fp] and the saved LR at
[fp + 8], so a frame whose fp is 0 sends the walker to address 0x8.
Fibers seeded exactly that. Nothing hit it until now because the walk only
happens when something captures a stack trace, and Zig's DebugAllocator --
which `zig test` uses as std.testing.allocator -- does so on EVERY allocation.
The same CLEAR source runs clean through `./clear run --safe`, which is why
ordinary fiber, stream and nested-BG programs all pass on macOS.
A fiber now parks a null frame record { saved_fp = 0, saved_lr = 0 } above its
initial SP -- where its own stack, growing down, never reaches -- and points fp
at it. The walk reads lr = 0 and stops cleanly.
Scoped to the aarch64 branch so x86-64 keeps its existing Context exactly;
fiber-core compiles for x86_64-linux, aarch64-linux and arm64-macos. The
isolated case goes from segfault to 11/11 passing.
Co-Authored-By: Claude Opus 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01KSuF9ba5qccHAU1gZc9oQ2
It described the unwinder hang and the translate-c spin as open problems and told people to exclude four specs. Both are fixed and nothing needs excluding: the full suite is 7910 examples, 0 failures. Kept the diagnosis of each, and the note that adding CFI did NOT fix the unwinder hang, since that is the obvious-looking wrong turn.
| # EINVAL -- the address-space cap is simply not enforced there, which is the | ||
| # same reason `ulimit -v` is rejected on that platform. Learn it once by | ||
| # trying, rather than hardcoding a platform list that would go stale. | ||
| @rlimit_as_usable = true |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.