Skip to content

Merge a game with its updates and DLC while packing, without copying - #141

Open
raiddevel wants to merge 3 commits into
PSBrew:mainfrom
raiddevel:layered-source
Open

raiddevel wants to merge 3 commits into
PSBrew:mainfrom
raiddevel:layered-source

Conversation

@raiddevel

Copy link
Copy Markdown

🤔 Why?

  • A game usually arrives in pieces: the base game, an update, sometimes a backport, and DLC, each one a folder or a RAR/ZIP.
  • Packing them meant copying every piece over the base folder first (a second full copy of the game on disk, a long wait, and the original folders modified), then running mkpfs.

🔧 What changed

  • mkpfs pack folder takes --layer PATH (repeatable, bottom to top). Each layer is a folder or a .rar/.zip; a file in a later layer replaces the same file in the layers below it, folders are merged, and the result is streamed straight into the usual exFAT-wrapped .ffpfsc builder.
  • Nothing is copied and no source folder is modified. The image is byte-for-byte the one you would get from the hand-merged folder.
  • The game folder inside each layer is detected (eboot.bin/sce_sys, a *-app folder, or the single folder inside); PATH::FOLDER picks it explicitly. Release notes (.nfo, .txt) next to it are not packed.
  • Emulation builds still get their ampr_emu.index, generated in memory from the merged listing; a stale index found in any layer is replaced.
  • --password applies to every encrypted archive among the layers. --raw is refused with layers (always exFAT-wrapped).

🧪 How to test

  • mkpfs pack folder ./Game ./Game.ffpfsc --layer ./Game-Update --layer ./Game-DLC.zip
  • mkpfs unpack --deep ./Game.ffpfsc ./out and compare with the folder you get by copying the update and the DLC over the game.
  • uv run --frozen pytest tests/mkpfs/test_layered_source.py (folder+folder, folder+zip, zip base + folder layer, file/folder replacement, AMPR index, CLI round trip, and "the exFAT image of the layers equals the one of the hand-merged folder").
  • Builds on the archive-source pull request (same ArchiveSubtree streaming, now behind a small SourceTree interface shared by archives and layers).

💬 Notes for non-technical readers

  • Packing a single folder or a single archive is unchanged. Layers only come into play when --layer is given.
  • The order matters: the last --layer is on top. Put the update after the game, and the DLC after the update, unless the DLC documentation says otherwise.
  • A DLC that ships its own eboot.bin replaces the one below it, exactly as it would if you copied the folders by hand; mkpfs warns when the merged view ends up without eboot.bin + sce_sys/param.json at its root.
  • The GUI does not expose layers yet; this pull request is CLI and library only.
  • Used in the field by ps5-game-packer (Windows/Linux front end, GPL-3.0), which proposes the layer order from the file names and contents and lets the user reorder them before calling mkpfs.

Builds on #139 (its commits show here until that one is merged). Related to #138.

Game releases usually arrive as a RAR or ZIP. Until now they had to be
extracted first, which costs time, a full extra copy of the game on disk,
and a lot of manual steps for large titles.

`pack folder` now accepts a `.rar` or `.zip` as the source. The archive is
listed once, the game folder inside it is located automatically, and each
member is streamed into the exFAT-wrapped `.ffpfsc` builder. Nothing is
extracted and no temporary copy is written; the resulting image is the
same one the extracted folder would produce.

- RAR via the `unrar` tool (multi-volume sets and passwords supported),
  ZIP built in (AES with the optional `pyzipper`, extra `mkpfs[archives]`).
- `--password` / `MKPFS_ARCHIVE_PASSWORD` and `--archive-root` for
  archives that hold several games.
- The AMPR emulation index is generated in memory for emulation builds.
- `batch` picks up archives next to folders and images.
Newer UnRAR builds (7.30) list the archive's quick-open record as a
member of type Service. It was counted as a file, so archive listings
showed one entry too many. Only File and Directory entries are members now.
A game, its update and its DLC usually arrive as separate folders or
archives. Packing them meant copying every piece over the base folder
first, which needs a second copy of the game and modifies the originals.

pack folder now takes --layer (repeatable, bottom to top) with folders or
RAR/ZIP archives. The listings are merged, the highest layer wins for each
path, and contents are streamed from whichever layer provides them. The
image equals the one built from the hand-merged folder, and the AMPR index
of emulation builds is generated in memory from the merged listing.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant