Skip to content

[Due for payment 2026-09-02] [No QA] Prevent sticky disks from growing unboundedly #98593

Closed

Description

@mountiny

Problem

A Blacksmith sticky disk is an ext4 block device, not a tarball, and the sticky disk action commits the mount in its post-job phase. Every job therefore snapshots whatever it wrote into node_modules before it ended — ESLint/Babel/rspack caches in node_modules/.cache, and Gradle/CMake output for native React Native libraries in node_modules/<lib>/android/build and .cxx. Because commit defaults to true, each run layered its generated files onto the previous snapshot and the next run started from that fatter disk.

The growth compounds worse than the file listing suggests: freed ext4 blocks are never returned to the storage layer, because fstrim on a sticky disk mount fails with the discard operation is not supported. A node_modules that is 3.8 GB after a clean npm ci reported 30 GB of live files and 103 GB of allocated blocks after about 3 weeks on the same key. At $0.50/GB/month, one key cost roughly $50/month.

Solution

Separate who produces the snapshot from who consumes it. setupNode gains a SEED_STICKY_DISKS input that selects the commit mode of the node_modules disks. A new seedStickyDisks.yml workflow passes SEED_STICKY_DISKS: 'true' to get commit: if-missing, installs dependencies and does nothing else, so its snapshot is a clean npm ci result by construction. It runs on every pull request and every push to main, and is a no-op once a key already has a snapshot.

Every other workflow keeps the default and mounts with commit: false. Jobs still read node_modules from the snapshot and may write caches, Gradle output and .cxx into their own clone, but the clone is discarded on unmount instead of committed, so disks stay at clean-install size permanently. The ~/.npm disk keeps committing with commit: on-change, since its key carries no hash and it must absorb new tarballs across lockfile bumps. Cache-hit detection is unchanged: the node_modules/.stickydisk-installed marker is written at the end of setupNode, so it lands in the seed snapshot and consumers skip npm ci when they see it.

PR

#98422

Issue OwnerCurrent Issue Owner: @collectioneur

Activity

  1. changed the title [-][No QA] Prevent sticky disks from growing unboundedly[/-] [+][Due for payment 2026-08-28] [No QA] Prevent sticky disks from growing unboundedly[/+] on Aug 21, 2026
  2. melvin-bot commented on Aug 21, 2026

    @melvin-bot

    @mountiny @collectioneur

    The solution for this issue has been 🚀 deployed to production 🚀 in version 9.4.57-3 and is now subject to a 7-day regression period 📆. Here is the list of pull requests that resolve this issue:

    If no regressions arise, payment will be issued on 2026-08-28. 🎊

    The following checklist (instructions) will need to be completed before the issue can be closed. Please copy/paste the Contributor+ Checklist from here into a new comment on this GH and complete it. If you have the K2 extension, you can simply click: [this button]. If no checklist is needed for this issue, you can click: [no checklist button]

  3. changed the title [-][Due for payment 2026-08-28] [No QA] Prevent sticky disks from growing unboundedly[/-] [+][No QA] Prevent sticky disks from growing unboundedly[/+] on Aug 21, 2026
  4. mountiny commented on Aug 21, 2026

    @mountiny
    ContributorAuthor

    We still need to follow up

  5. melvin-bot commented on Aug 24, 2026

    @melvin-bot

    @collectioneur Uh oh! This issue is overdue by 2 days. Don't forget to update your issues!

  6. changed the title [-][No QA] Prevent sticky disks from growing unboundedly[/-] [+][Due for payment 2026-09-02] [No QA] Prevent sticky disks from growing unboundedly[/+] on Aug 26, 2026
  7. melvin-bot commented on Aug 26, 2026

    @melvin-bot

    @mountiny @collectioneur

    The solution for this issue has been 🚀 deployed to production 🚀 in version 9.4.61-4 and is now subject to a 7-day regression period 📆. Here is the list of pull requests that resolve this issue:

    If no regressions arise, payment will be issued on 2026-09-02. 🎊

    The following checklist (instructions) will need to be completed before the issue can be closed. Please copy/paste the Contributor+ Checklist from here into a new comment on this GH and complete it. If you have the K2 extension, you can simply click: [this button]. If no checklist is needed for this issue, you can click: [no checklist button]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions