Skip to content

feat: FHIR Bulk Data $import and CLI backup/restore - #56

Merged
degoke merged 5 commits into
mainfrom
degoke/import-backup-86c6
Sep 21, 2026
Merged

degoke merged 5 commits into
mainfrom
degoke/import-backup-86c6

Conversation

@degoke

@degoke degoke commented Sep 21, 2026

Copy link
Copy Markdown
Owner

Adds FHIR Bulk Data $import and CLI backup / restore.

Merged main after #50#53, #51, and #55. Conflicts were overlapping CapabilityStatement/docs edits: $import is now a capabilityFlags.BulkImport system operation next to $export, not a second marshalCapabilityStatement signature.

HTTP $import

  • POST /fhir/$import with Prefer: respond-async (also accepts respond-async, wait=…) and a Parameters body
  • Each input part needs type plus inline NDJSON (valueString) or a url
  • Poll GET /fhir/$import/status/{jobId}; cancel with DELETE
  • Error NDJSON: GET /fhir/$import/files/{jobId}/{file}
  • NDJSON lines are created or updated through core after stripping meta.versionId / lastUpdated (same as CLI restore)
  • Kickoff and cancel require write authorization per input type, not bulk-export read
  • Advertised on CapabilityStatement only when BulkImportService is wired
  • Invalid Parameters map to 400; cancel is not overwritten by a later complete

CLI

haistack backup [dir]    # default dir: backup
haistack restore [dir]

Backup writes {ResourceType}.ndjson plus manifest.json. Restore upserts those files and strips meta.versionId so core assigns new versions.

Runtime

pkg/runtime registers jobs.TypeImportBulk (export.import) alongside bulk export. Job/file stores remain in-memory (same as $export). Runtime wires HTTPLoader for http(s) input.url (file/data schemes rejected; 60s timeout, 64MiB cap).

Review follow-ups

  • In-memory job Update will not un-cancel a cancelled job
  • Write auth is checked per input resource type
  • HTTPLoader Close is checked for errcheck CI
  • After main: $import is advertised via capabilityFlags.BulkImport / systemOperations
Open in Web Open in Cursor 

cursoragent and others added 4 commits September 21, 2026 09:37
Add async POST /$import (Prefer: respond-async) that reads Parameters
with inline NDJSON, upserts resources, and polls at /$import/status/{id}.
Wire the service in runtime. CLI backup writes per-type NDJSON plus a
manifest; restore reloads that directory with create-or-update.

Co-authored-by: Adegoke Adewoye <[email protected]>
Match CLI restore by clearing server-assigned meta before persist, require
write authorization on kickoff instead of export-read, map client kickoff
errors to 400, keep cancelled jobs from completing, parse Prefer respond-async
comma form, and add GET $import/files for error artifacts.

Co-authored-by: Adegoke Adewoye <[email protected]>
…oader

Keep cancelled in-memory jobs from being overwritten by complete/in-progress
updates. Authorize create/update on each input resource type at kickoff.
Wire a default HTTP(S) URLLoader with timeout, size limit, and scheme checks.

Co-authored-by: Adegoke Adewoye <[email protected]>
golangci-lint errcheck failed CI because Load deferred Close without
reading the error. Ignore the close error after the body is consumed.

Co-authored-by: Adegoke Adewoye <[email protected]>
@degoke
degoke marked this pull request as ready for review September 21, 2026 12:21
Advertise $import through capabilityFlags/systemOperations alongside
export instead of a separate marshalCapabilityStatement signature.
Keep Patient $export and $import routes in HTTP docs.

Co-authored-by: Adegoke Adewoye <[email protected]>
@degoke
degoke merged commit 439d538 into main Sep 21, 2026
3 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants