Repository navigation
Write exports through storage instead of a disk - #47
Merged
Merged
Conversation
Exports were the last part of the job pipeline that opened a filesystem: the finished document went to blobRoot/exports/... with writeFile, and a PDF read its stills back off disk. There is no `node:fs` in that file at all now. Every path involved was already a key. The still a report waits on is requested at a key the export chooses, and the result is stored at a key the job row records, so nothing here needed a filesystem in the first place -- it just happened to have one. runDueExport takes one queued export and runs it, claiming by conditional update so two drivers racing cannot both take the same row. The node pump's export slot calls it rather than keeping its own copy of the query. Not yet wired into the Workers cron, and the reason is recorded there: exports still import the PDF report builder and the SVG compositor from @onelight/worker, whose index reaches media.ts and therefore node:child_process. Mounting them on that target means lifting those two out of the media graph first, exactly as the rendition planner was lifted. A comment report requested there stays queued until then.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Exports were the last part of the job pipeline that opened a filesystem. There is no
node:fsimport in that file at all now.What changed
writeFile(blobRoot/...)readFile(stillsDir/...), and deletes them by key afterwardsrunDueExporttakes one queued export and runs it, claiming by conditional update so two drivers racing cannot both take the same rowEvery path involved was already a key. The still a report waits on is requested at a key the export chooses, and the result is stored at a key the job row records. Nothing here needed a filesystem — it just happened to have one, because it was written when there was only one target.
The node pump's export slot now calls the shared runner rather than keeping its own copy of the claim query, so the two cannot drift.
Not yet on the Workers target, and why
Exports still import
buildPdfReportand the SVG compositor from@onelight/worker, whose index reachesmedia.tsand thereforenode:child_process. Mounting exports on Workers means lifting those two out of the media graph first — exactly the move made for the rendition planner in #19.That is a separate piece of work and it is recorded in
apps/cf/src/index.tsrather than left to be rediscovered. A comment report requested on that target stays queued until then.Context
This is the fourth thing that exists on the node target and was never wired for Workers: after the API being shadowed by asset routing, password hashing being impossible, and the sweeps never running. Exports differ in that the code genuinely could not have worked there — it wrote to a disk that does not exist — where the others were wiring.
Gates: typecheck, eslint, prettier, full vitest (1023 passed).