Store pairing history chunks on the identity service's disk (GRYT-1591)

[sivert-io]

Step 4 of the build plan in docs/pairing-design.md (Gryt-chat/crypto). The pairing relay from #45 can now hold history chunks, so core's history transfer (Gryt-chat/core#33) has somewhere to send them. Task: GRYT-1591.

All of it is review-required (packages/auth/**). Nothing here touches Keycloak.

What it does

Three endpoints under /api/v1/pairing/sessions/:id/chunks/:n. They match core's relay client and its fake:

• PUT, from A only, and only after the reveal. The body is {"body": "<base64url>"}, up to 2 MiB once decoded. Answers 201 {}. A slot that's already used gets 409 exists, unless it's the same bytes again, which counts as a retry and gets 201.
• GET, from N only. Answers {"body"}, or 404 not_found. The second fetch deletes the chunk.
• DELETE, from N only. Answers 204, even when the chunk's already gone.

Chunks are saved as raw bytes under $GRYT_IDENTITY_DATA_DIR/pairing/<session>/<n>, with 0600 files in 0700 folders. They're never parsed and never logged. The logs only get a count and a MiB total once a minute, plus refusals by reason.

A chunk goes when N deletes it or fetches it twice, when its session closes or expires (an hour after the claim at most), or an hour after upload. The sweep checks all three, and every minute it also removes any folder that doesn't belong to a live session.

Limits. The first four are environment variables, in MiB:

	Default	Answer
Per session, in sealed bytes	256 MiB	507 session_full
In total, with each file counted as 4 KiB at least	2 GiB	507 full
Free space left on the volume after the write	5 GiB	507 disk_low
Uploads per address per day	1 GiB	429 rate_limited
One chunk	2 MiB	413 too_large
Slot numbers	0 to 65535	400 invalid_slot

Core's sender treats any 507 as the end of what fits and marks the history as cut short, which is the design's behaviour. Free space comes from statfs on the chunk folder. For a named volume that's the Docker disk, the same one Postgres is on.

What happens on a restart mid-transfer

The sessions live in memory, so they're gone. At startup the service empties data/pairing/, removing only folders named like a session id, so ca-key.pem next to it stays. After that, every call on the old session gets 404 not_found, because the relay has no memory of it and can't answer 410.

By then keys and sign-in are already done, because history comes after N commits. So on the phone or desktop:

• A's next PUT gets a 404. That isn't a 507, so the sender throws and A's history send ends.
• N gets 404 for each chunk it has listed but not fetched yet. It counts them as missing, then its message poll gets a 404 too, and the pairing ends there.
• N keeps whatever it already stored. Linking again from the same device sends the rest, and N merges by message id, so nothing is doubled.

That's the design's "anyone halfway through starts again". The test a restart wipes the chunks of sessions it no longer knows, and nothing else covers the relay's half of it.

What to look at

• ChunkStore.put in chunks.ts, and the order of its checks. The caps are reserved before the first await, so two uploads at once can't both squeeze under the same limit. If the free-space check or the write fails, the reservation goes back. If the session ends while a write is in flight, the file gets removed after the write. I'm least sure about this part.
• The wipe and the orphan sweep delete folders. Both only touch names matching ^[0-9A-HJKMNP-TV-Z]{26}$ directly under data/pairing, and the folder is always <data dir>/pairing, not configurable, so a bad setting can't point it at the CA key.
• Accepting an identical retry is a change from core's fake, which answers 409 exists. Core's sender only retries after a network error, when the first PUT may well have landed, so a 409 there would end the history send for no reason.
• The body limit is 2 MiB of base64 (about 2.8 MB) on chunk paths only. Every other pairing route keeps the envelope limit.
• Relay changes. end() now drops the session's chunks, and sweep() returns a promise so the tests can wait for the disk.

Tests

identity/test/chunks.test.ts, 13 tests, against a real temp folder and a faked free-space check. They cover upload, fetch and delete, the second fetch deleting a chunk, the retry, sides and states, a token from another session (401), session close, expiry and the one-hour sweep, each cap, the free-space refusal, the per-address limit, the restart wipe, the env parsing and the body limit. yarn build and yarn test pass, with all 24 tests green.

I also ran core's src/pairing/history.test.ts (core main 446c63f) against this relay, built and serving on localhost, with only the relay swapped out. Five of its seven tests apply, and all five pass: 5,000 messages over 1,202 chunk uploads and 1,202 fetches, with nothing left on disk at the end. The other two need a relay that tampers with chunks, so they only make sense with the fake. Two of the test's assertions depend on the fake's timing rather than on the relay. Those are in GRYT-1594.

Deploying

The relay from #45 isn't deployed yet, so this deploys both. It's Sivert's step. On dev.lan, in packages/auth:

1. git pull
2. Check GRYT_IMPORT_REALM=0 in .env
3. df -h and make sure there's room for the build plus the 5 GiB floor
4. docker compose -f docker-compose.keycloak.yml -p auth up -d --no-deps --build identity

There's no new volume. The chunks go on gryt-identity-data, which is already mounted at /app/data and owned by the gryt user from the Dockerfile. The four GRYT_PAIRING_CHUNKS_* variables have their defaults in compose, so .env only needs them to change a limit. GRYT_PAIRING_TRUSTED_PROXIES from #45 still has to be set, or the 1 GiB a day is shared by everyone behind the tunnel.

🤖 Generated with Claude Code