Repository navigation
Restore as many files at a time as the drive does best, with progress - #55
Open
ChakraFusion wants to merge 1 commit into
Open
ChakraFusion wants to merge 1 commit into
ChakraFusion wants to merge 1 commit into
Conversation
Putting a large save back - a quarter of a million files - wrote them one
after another, looking up each one's folder again, while the drive sat at
90% active time and 15 MB/s, and the window showed a spinner for many
minutes with nothing to say what it was doing.
- internal/iopar runs many small file operations as many at a time as
suits the drive they are on: one at a time on a hard disk, starting at
8 on a SATA SSD and 32 on NVMe (Windows: the volume's seek penalty and
bus type; Linux: rotational and nvme). On an SSD the number then moves
every 750 ms the way that finishes more - on while it helps, back when
it hurts, and now and then the other way once settled, so a drive that
slows as it goes is followed.
- A restore makes every folder once, then writes the files through it;
checking the archive reads its entries through it too. On a 4-thread
machine restoring Project Zomboid (238,410 files) the processor, not the
SSD, is the limit: writing went from 2m29s to 1m42s, settling at 3-8 at
once; machines with more cores go further by themselves.
- A restore says how far it is - rebuilding a compacted snapshot,
checking it, writing the files - with percent and, once it can be told,
the time left, in the game's header ("restoring" in the game payload).
The drive-adaptation test is timed with sub-millisecond sleeps, so it
gets three runs and one has to settle near the best; a broken adaptation
misses in all three.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
|
@ChakraFusion is attempting to deploy a commit to the sivadaboi's projects Team on Vercel. A member of the Team first needs to authorize it. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
As offered in #21.
internal/ioparruns many small file operations at whatever concurrency suits the drive: 1 on an HDD, starting at 8 on a SATA SSD and 32 on NVMe, then adapting every 750 ms. Drive type comes from the seek penalty and bus type on Windows, and fromrotational/nvmeon Linux.iopar. Restoring Project Zomboid (238,410 files) on a 4-thread PC: 2 min 29 s → 1 min 42 s.restoringin the game payload).Tests:
iopar(settles near the drive's best, three runs since it's timed with sub-millisecond sleeps; HDD one at a time; every item once with monotonic progress; stops at the first error); frontend progress formatting.🤖 Generated with Claude Code