Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,11 @@

## Unreleased

- Raised llama-server startup health timeout to 5 minutes for cold large-model loads while still failing fast when the runtime process exits before readiness.
- Added Apple Silicon host support: `thirdshift doctor` now passes on `darwin/arm64`, detects the GPU via `system_profiler` and unified memory via `sysctl`, and reports vendor `apple`. macOS uses the existing signed `darwin/arm64` llama.cpp build, which has Metal compiled in, so no new runtime artifact was needed.
- Apple Silicon reports a GPU working-set budget of 65% of unified memory in place of discrete VRAM, which feeds the VRAM doctor check and auto model selection.
- Added `apple` to the `gpu_vendor` enum in the `node.heartbeat` schema; without it the coordinator rejected every Apple heartbeat.

- Fixed `thirdshift start` with no `--model` failing on installed binaries: automatic selection listed the catalog directory from disk instead of falling back to the catalog embedded in the binary.

- Fixed nodes going permanently offline about an hour after starting: access tokens were fetched once at startup and never renewed, so any reconnect past the token lifetime was rejected with 401 until the process was restarted by hand. The agent now renews proactively near expiry and retries once on a 401.
Expand Down
2 changes: 1 addition & 1 deletion docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ The current repository contains the contracts, database schema, coordinator heal
The Milestone 1 node code is split behind interfaces so macOS development can use fakes while Windows-specific host concerns remain build-tagged.

- Doctor checks report platform, GPU vendor, VRAM, RAM, disk, and configured outbound HTTPS/WSS reachability. Non-Windows hosts report unsupported host status without crashing.
- GPU vendor detection probes `nvidia-smi` first and falls back to `Win32_VideoController` through PowerShell on Windows, yielding `nvidia`, `amd`, or `unknown`. AMD hosts pass and serve on llama.cpp's Vulkan backend. `AdapterRAM` is a 32-bit field, so VRAM that lands in the wrap band is reported unknown rather than estimated.
- GPU vendor detection probes `nvidia-smi` first, then `system_profiler SPDisplaysDataType` on macOS and `Win32_VideoController` through PowerShell on Windows, yielding `nvidia`, `amd`, `apple`, or `unknown`. AMD hosts pass and serve on llama.cpp's Vulkan backend. `AdapterRAM` is a 32-bit field, so VRAM that lands in the wrap band is reported unknown rather than estimated.
- Runtime manifests are signed with Ed25519. The manager verifies the manifest, downloads artifacts to a temporary path, checks SHA-256 and byte length, extracts archives, promotes into a versioned runtime directory, and tracks previous/current installs for rollback.
- Artifact keys may carry a compute backend suffix (`windows/amd64/cuda`, `windows/amd64/vulkan`). The node resolves its vendor to a backend and tries the backend key first, falling back to the bare platform key. Each key installs into its own directory, so a CUDA and a Vulkan build of one release never collide. The coordinator records every key's hash for a release and treats runtime-hash validity as set membership, so a Vulkan host is schedulable with no scheduler change.
- Model cache downloads by HTTP Range into `<sha256>.partial`, verifies byte length and SHA-256, atomically promotes to `<sha256>.gguf`, and applies LRU eviction without removing the active model.
Expand Down
5 changes: 5 additions & 0 deletions docs/DECISIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -117,3 +117,8 @@
| D-109 | The public ticker shows a host once it has earned, or while it is connected | The previous rule — any node seen in the last 24h — kept dead test registrations on the launch page for a day. A machine that registered, never served, and went away is an artifact, not a contributor. Hosts who have earned stay visible while asleep, so a real host's credit does not vanish when they close the lid |
| D-110 | The agent renews its own access token: proactively near expiry, and reactively on a 401 | Access tokens live an hour and were fetched once at `thirdshift start`. A node that lost its session after that hour reconnected with a dead token and was rejected forever — a host left running overnight was silently offline by morning, needing a manual restart nobody would know to perform. The agent already holds the node private key, so it can sign its own renewal. Reactive 401 retry covers tokens whose expiry is unknown; proactive renewal avoids the disconnect entirely |
| D-111 | Catalog listing falls back to the embedded catalog, not just per-file reads | `ReadCatalogFile` fell back to the embedded catalog but `LoadSelectableManifests` listed the directory directly, so auto model selection failed on every installed binary with "read catalog dir: cannot find the path". The same omission as the protocol schemas in D-0xx: an embedded fallback is only complete when *enumeration* is covered too, not just reads by name |
| D-112 | Apple Silicon Macs are supported hosts, expanding past handoff 5.3 and D-002 alongside the AMD addition | The signed `darwin/arm64` llama.cpp artifact has been the dev and test path since Milestone 1, so this un-gates an already-exercised runtime rather than integrating a new one. Supply remains the binding constraint and Macs are plentiful |
| D-113 | macOS uses the bare `darwin/arm64` artifact key; no `darwin/arm64/metal` key exists | Metal is compiled into the single macOS llama.cpp build, so there is nothing to select between. `BackendForVendor` maps `apple` to an empty backend so darwin never probes a suffixed key that cannot exist, while unknown and AMD vendors keep taking Vulkan on Windows |
| D-114 | Apple Silicon reports a GPU budget of 65 percent of unified memory, not a VRAM figure | Apple Silicon has no discrete VRAM: CPU and GPU share one pool, and Metal's own recommended working set on a 16 GB machine is roughly 70 percent. We sit below that on purpose because a host Mac is somebody's daily-driver laptop, and auto-selection must not pick a model that makes the machine unusable while it serves. On a 16 GB M4 this yields 10649 MB and selects the 7B tier; on 64 GB it yields 42598 MB and reaches the 32B tier. Hosts who want to push further can pin a larger model with `--model` |
| D-115 | `gpu_vendor` gained `apple` in the heartbeat schema enum | The field was additive but its enum was closed to nvidia, amd and unknown. Without this the coordinator's protocol validator rejects every Apple heartbeat and closes the session, so Apple support is impossible without the schema change |
| D-116 | Runtime startup health waits up to 5 minutes but exits fast when the child process dies | Cold GGUF loads can spend minutes compiling GPU shaders and paging several GB into unified memory or VRAM; a long ceiling costs nothing once warm because `/health` returns promptly, while process-exit detection keeps real crashes from waiting out the ceiling |
1 change: 1 addition & 0 deletions docs/LAUNCH_CHECKLIST.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
- [x] Hash verification failure blocks execution.
- [x] Scheduler placement is in-process and alpha-scale.
- [x] Windows 11 x64 NVIDIA host verified on real hardware (2026-07-31, RTX 3060 Ti).
- [ ] Apple Silicon host verified on real hardware (Metal backend). `thirdshift doctor --json` has been run on an M4 (16 GB) and reports platform pass, vendor apple, and a 10649 MB GPU budget; still unverified end to end: `thirdshift start` auto-selecting and serving a routed request on Metal.
- [ ] Windows 11 x64 AMD host verified on real hardware (Vulkan backend). Unverified until a Radeon machine runs `thirdshift doctor` and completes a routed request: vendor detection, Vulkan artifact download, and llama-server startup on Vulkan have only been exercised against fixtures.
- [ ] Five stable cafe nodes enrolled. Pending human pilot.
- [ ] At least 70% scheduled-hours connected. Pending pilot measurement.
Expand Down
236 changes: 236 additions & 0 deletions internal/node/hardware/apple_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,236 @@
package hardware

import (
"context"
"errors"
"testing"
)

// Captured verbatim from `system_profiler SPDisplaysDataType -json` on an
// Apple M4 MacBook Pro (10-core GPU, 16 GB unified memory), trimmed to the
// fields this code reads plus enough context to keep the shape realistic.
const spDisplaysM4 = `{
"SPDisplaysDataType" : [
{
"_name" : "Apple M4",
"spdisplays_mtlgpufamilysupport" : "spdisplays_metal4",
"spdisplays_vendor" : "sppci_vendor_Apple",
"sppci_bus" : "spdisplays_builtin",
"sppci_cores" : "10",
"sppci_device_type" : "spdisplays_gpu",
"sppci_model" : "Apple M4"
}
]
}`

// A larger Apple Silicon machine, for the 32B tier.
const spDisplaysM3Max = `{
"SPDisplaysDataType" : [
{
"_name" : "Apple M3 Max",
"spdisplays_vendor" : "sppci_vendor_Apple",
"sppci_bus" : "spdisplays_builtin",
"sppci_cores" : "40",
"sppci_device_type" : "spdisplays_gpu",
"sppci_model" : "Apple M3 Max"
}
]
}`

// An Intel Mac with a discrete AMD card: not Apple Silicon.
const spDisplaysIntelAMD = `{
"SPDisplaysDataType" : [
{
"_name" : "AMD Radeon Pro 5500M",
"spdisplays_vendor" : "sppci_vendor_amd",
"sppci_device_type" : "spdisplays_gpu",
"sppci_model" : "AMD Radeon Pro 5500M"
}
]
}`

const (
memsize16GB = "17179869184\n"
memsize64GB = "68719476736\n"
)

type appleRunner struct {
displays string
displaysErr error
memsize string
memsizeErr error
calls []string
}

func (r *appleRunner) Run(_ context.Context, name string, _ ...string) ([]byte, error) {
r.calls = append(r.calls, name)
switch name {
case "nvidia-smi":
return nil, errors.New("exec: nvidia-smi not found")
case "system_profiler":
return []byte(r.displays), r.displaysErr
case "sysctl":
return []byte(r.memsize), r.memsizeErr
default:
return nil, errors.New("unexpected command " + name)
}
}

func TestParseAppleDisplaysJSON(t *testing.T) {
gpus, err := ParseAppleDisplaysJSON(spDisplaysM4)
if err != nil {
t.Fatalf("parse: %v", err)
}
if len(gpus) != 1 {
t.Fatalf("gpus = %#v", gpus)
}
if gpus[0].Model != "Apple M4" || gpus[0].Cores != "10" {
t.Fatalf("gpu = %#v", gpus[0])
}
if _, err := ParseAppleDisplaysJSON(" "); err == nil {
t.Fatal("empty output accepted")
}
if _, err := ParseAppleDisplaysJSON("{}"); err == nil {
t.Fatal("a report with no GPUs was accepted")
}
}

func TestSelectAppleGPUIgnoresNonApple(t *testing.T) {
apple, err := ParseAppleDisplaysJSON(spDisplaysM4)
if err != nil {
t.Fatalf("parse: %v", err)
}
if gpu, ok := SelectAppleGPU(apple); !ok || gpu.Model != "Apple M4" {
t.Fatalf("apple gpu = %#v ok=%v", gpu, ok)
}

// An Intel Mac with a discrete AMD card is not an Apple Silicon host.
intel, err := ParseAppleDisplaysJSON(spDisplaysIntelAMD)
if err != nil {
t.Fatalf("parse intel: %v", err)
}
if _, ok := SelectAppleGPU(intel); ok {
t.Fatal("a discrete AMD card on an Intel Mac was treated as Apple Silicon")
}
}

func TestParseMemsize(t *testing.T) {
got, err := ParseMemsize(memsize16GB)
if err != nil || got != 17179869184 {
t.Fatalf("memsize = (%d, %v)", got, err)
}
for _, bad := range []string{"", " ", "not-a-number", "0", "-1"} {
if _, err := ParseMemsize(bad); err == nil {
t.Fatalf("memsize %q was accepted", bad)
}
}
}

// The budget is a share of one shared pool, never the whole thing: the machine
// still has to run macOS while it serves.
func TestAppleGPUMemoryBudget(t *testing.T) {
for name, tc := range map[string]struct {
totalBytes int64
wantMB int64
}{
"16GB M4": {17179869184, 10649},
"64GB": {68719476736, 42598},
"8GB": {8589934592, 5324},
"unknown": {0, VRAMUnknown},
"nonsense": {-5, VRAMUnknown},
} {
t.Run(name, func(t *testing.T) {
if got := AppleGPUMemoryBudgetMB(tc.totalBytes); got != tc.wantMB {
t.Fatalf("budget = %d MB, want %d MB", got, tc.wantMB)
}
})
}

// Never hand the GPU the entire pool.
total := int64(17179869184)
if AppleGPUMemoryBudgetMB(total) >= total/1024/1024 {
t.Fatal("budget must be strictly less than total unified memory")
}
}

func TestDetectGPUVendorOnDarwin(t *testing.T) {
t.Run("apple silicon", func(t *testing.T) {
runner := &appleRunner{displays: spDisplaysM4, memsize: memsize16GB}
if got := DetectGPUVendor(context.Background(), runner, "darwin"); got != VendorApple {
t.Fatalf("vendor = %q, want apple", got)
}
})

t.Run("intel mac with amd card is not apple", func(t *testing.T) {
runner := &appleRunner{displays: spDisplaysIntelAMD}
if got := DetectGPUVendor(context.Background(), runner, "darwin"); got != VendorUnknown {
t.Fatalf("vendor = %q, want unknown", got)
}
})

t.Run("system_profiler failure is unknown, not a crash", func(t *testing.T) {
runner := &appleRunner{displaysErr: errors.New("not found")}
if got := DetectGPUVendor(context.Background(), runner, "darwin"); got != VendorUnknown {
t.Fatalf("vendor = %q, want unknown", got)
}
})

t.Run("darwin never probes windows WMI", func(t *testing.T) {
runner := &appleRunner{displays: spDisplaysM4, memsize: memsize16GB}
DetectGPUVendor(context.Background(), runner, "darwin")
for _, call := range runner.calls {
if call == "powershell" {
t.Fatal("WMI was probed on darwin")
}
}
})
}

func TestDetectHostResourcesOnAppleSilicon(t *testing.T) {
for name, tc := range map[string]struct {
memsize string
wantVRAMMB int64
wantName string
}{
"16GB M4": {memsize16GB, 10649, "Apple M4"},
"64GB M3 Max": {memsize64GB, 42598, "Apple M3 Max"},
} {
t.Run(name, func(t *testing.T) {
displays := spDisplaysM4
if tc.wantName == "Apple M3 Max" {
displays = spDisplaysM3Max
}
runner := &appleRunner{displays: displays, memsize: tc.memsize}
resources, err := DetectHostResources(context.Background(), runner, ".", "darwin")
if err != nil {
t.Fatalf("detect: %v", err)
}
if resources.GPUVendor != VendorApple {
t.Fatalf("vendor = %q", resources.GPUVendor)
}
if resources.GPUName != tc.wantName {
t.Fatalf("gpu name = %q, want %q", resources.GPUName, tc.wantName)
}
if resources.VRAMTotalMB != tc.wantVRAMMB {
t.Fatalf("gpu budget = %d MB, want %d MB", resources.VRAMTotalMB, tc.wantVRAMMB)
}
if !resources.VRAMKnown() {
t.Fatal("a measured unified-memory budget should count as known")
}
})
}

// Unified memory unreadable: vendor still apple, budget unknown, and the
// caller must not be handed a fabricated number.
runner := &appleRunner{displays: spDisplaysM4, memsizeErr: errors.New("boom")}
resources, err := DetectHostResources(context.Background(), runner, ".", "darwin")
if err != nil {
t.Fatalf("detect: %v", err)
}
if resources.GPUVendor != VendorApple {
t.Fatalf("vendor = %q, want apple", resources.GPUVendor)
}
if resources.VRAMKnown() {
t.Fatalf("budget = %d, want unknown when hw.memsize fails", resources.VRAMTotalMB)
}
}
81 changes: 80 additions & 1 deletion internal/node/hardware/doctor.go
Original file line number Diff line number Diff line change
Expand Up @@ -72,11 +72,18 @@ func checkPlatform() CheckResult {
Message: "Windows x64 host platform detected.",
}
}
if runtime.GOOS == "darwin" && runtime.GOARCH == "arm64" {
return CheckResult{
Name: "platform",
Status: StatusPass,
Message: "Apple Silicon host platform detected; llama.cpp will use the Metal backend.",
}
}
return CheckResult{
Name: "platform",
Status: StatusUnsupported,
Message: fmt.Sprintf("%s/%s is not a supported host platform for v0.1.", runtime.GOOS, runtime.GOARCH),
Remediation: "Run the host node on Windows 11 x64 with an NVIDIA or AMD GPU. macOS can still run local fake and CPU demo paths.",
Remediation: "Run the host node on Windows 11 x64 with an NVIDIA or AMD GPU, or on an Apple Silicon Mac.",
}
}

Expand All @@ -88,6 +95,12 @@ func checkGPU(ctx context.Context, opts DoctorOptions) ([]GPU, string, []CheckRe
if len(gpus) > 0 {
return gpus, VendorNvidia, checks
}
if runtime.GOOS == "darwin" {
if appleGPUs, appleChecks, ok := checkApple(ctx, opts); ok {
return appleGPUs, VendorApple, appleChecks
}
return gpus, VendorUnknown, checks
}
if runtime.GOOS != "windows" {
return gpus, VendorUnknown, checks
}
Expand All @@ -98,6 +111,72 @@ func checkGPU(ctx context.Context, opts DoctorOptions) ([]GPU, string, []CheckRe
return amdGPUs, VendorAMD, amdChecks
}

// checkApple reports the Apple Silicon GPU and its share of unified memory.
// There is no discrete VRAM here: the budget is a conservative fraction of the
// one shared pool, so the number is a working-set allowance rather than a
// hardware capacity.
func checkApple(ctx context.Context, opts DoctorOptions) ([]GPU, []CheckResult, bool) {
appleGPUs, err := DetectAppleGPUs(ctx, opts.Runner)
if err != nil {
return nil, nil, false
}
primary, ok := SelectAppleGPU(appleGPUs)
if !ok {
return nil, nil, false
}
name := strings.TrimSpace(firstNonEmptyString(primary.Model, primary.Name))

totalBytes, memErr := DetectAppleUnifiedMemoryBytes(ctx, opts.Runner)
budgetMB := VRAMUnknown
if memErr == nil {
budgetMB = AppleGPUMemoryBudgetMB(totalBytes)
}

gpuCheck := CheckResult{
Name: "gpu",
Status: StatusPass,
Message: fmt.Sprintf("%s detected; llama.cpp will use the Metal backend.", name),
Details: map[string]any{
"vendor": VendorApple,
"backend": "metal",
"cores": strings.TrimSpace(primary.Cores),
},
}

if budgetMB <= 0 {
return []GPU{{Name: name, VRAMTotalMB: VRAMUnknown}}, []CheckResult{
gpuCheck,
{
Name: "vram",
Status: StatusWarn,
Message: "Unified memory could not be read, so the GPU working-set budget is unknown.",
Remediation: "Confirm `sysctl -n hw.memsize` works on this host.",
Details: map[string]any{"vendor": VendorApple},
},
}, true
}

vram := CheckResult{
Name: "vram",
Status: StatusPass,
Message: fmt.Sprintf("Apple Silicon shares %0.1f GB unified memory; %0.1f GB (%d%%) is budgeted for the GPU.",
bytesToGiB(uint64(totalBytes)), float64(budgetMB)/1024, AppleGPUMemoryBudgetPercent),
Details: map[string]any{
"vendor": VendorApple,
"unified_memory_gb": bytesToGiB(uint64(totalBytes)),
"gpu_budget_gb": float64(budgetMB) / 1024,
"budget_percent": AppleGPUMemoryBudgetPercent,
"minimum_gb": bytesToGiB(MinimumVRAMBytes),
},
}
if uint64(budgetMB)*1024*1024 < MinimumVRAMBytes {
vram.Status = StatusFail
vram.Remediation = "Use an Apple Silicon Mac with more unified memory; the GPU budget is below the 8 GB floor."
}

return []GPU{{Name: name, VRAMTotalMB: budgetMB}}, []CheckResult{gpuCheck, vram}, true
}

// checkAMD reports whether an AMD adapter was found, and the checks to use in
// place of the NVIDIA failure pair when one was.
func checkAMD(ctx context.Context, opts DoctorOptions) ([]GPU, []CheckResult, bool) {
Expand Down
Loading