perf: cut full-run wall time ~8x (65s → 8s) - #38
Merged
Conversation
Profiling a full `apps -a` run (239 apps) showed the pipeline was dominated by a single blocking subprocess and a handful of avoidable serial paths. - Mac software update: `softwareupdate --list --all` blocked the entire scan for ~48s hitting Apple's servers; switch to `--no-scan` (reads macOS's own cached scan, effectively instant). Single biggest win. - CheckOrchestrator: scanner check groups ran strictly serially despite the docs claiming concurrency; fan them out through the existing bounded-channel WhenAll helper. Counting/mutation stays on the single reader. - Mac cask fallback: replaced O(N²) unresolved-set recomputation with HashSet lookups and raised the homebrew-api client concurrency 4 → 12. - ProcessRunner: subprocess cap 6 → max(6, ProcessorCount); child work is I/O-bound so it under-utilised multi-core machines. - ScannerHelper.FindExecutable: memoized (was re-walking PATH + 9 probes per call). - GoScanner: `go version -m` now runs concurrently per binary instead of serially. - PlistReader: dropped a dead per-plist MemoryStream allocation; added a fast-path for the common `_MASReceipt` location to avoid a full recursive bundle walk per app. - ProjectManifestFinder: parse each `.gitignore` once and cache it instead of re-reading per subdirectory. Output unchanged (239 discovered, 244 checked, 9 updates). All 286 tests pass. Co-Authored-By: Claude Opus 4.8 <[email protected]>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Profiled a full
apps -arun (239 apps) and removed the bottlenecks. Wall time dropped from ~65s to ~8s (≈8×). Output is byte-for-byte the same (239 discovered · 244 checked · 9 updates) and all 286 tests pass.The run was dominated by one blocking subprocess; the rest were serial paths, an O(N²) set recompute, an unbounded-latency subprocess cap, and a few avoidable allocations / repeated I/O.
Where the time went (measured, before)
softwareupdate --list --allbrew info --jsonEnhancements
MacApplicationsScannersoftwareupdate --list --all→--list --no-scan(read macOS's cached scan instead of blocking on Apple)CheckOrchestratorWhenAllhelper (were strictly serial despite docs)MacApplicationsScannerWhere(!resolved.Any(…))+Except→HashSetlookupsMacOsRegistrationhomebrew-apiclient concurrency4 → 12ProcessRunner6 → max(6, ProcessorCount)ScannerHelper.FindExecutableConcurrentDictionary)PATH+ 9File.Existsprobes on every callGoScannergo version -mper binary now runs concurrently (was serialawaitin a loop)PlistReadernew MemoryStream(Encoding.UTF8.GetBytes(xml))(allocated, never used)PlistReader.HasAppStoreReceiptContents/_MASReceipt/receiptlocation before falling back to a recursive walkAllDirectoriesbundle walk per app (common case)ProjectManifestFinder.gitignoreonce and cache; lookups viaHashSet.gitignoreonce per subdirectoryVerification
dotnet build— 0 warnings, 0 errorsdotnet test— 286 / 286 passedapps -a— 239 discovered · 244 checked · 9 updates (unchanged); ~8.2s vs ~65s baselineNotes / out of scope
AddAuditComponent) but never invoked —Orchestrator.cs:78is still a// TODO. Left untouched here; it's dead code, not a live perf cost. Wire it or remove it in a follow-up.bin/Release/net10.0on the dev machine is root-owned (leftover from a priorsudopublish/--install) and blocks Release/AOT builds locally; verified against the Debug build. Unrelated to this change.🤖 Generated with Claude Code