Skip to content

fix(record): include branch in the cell directory path - #162

Open
TingPing wants to merge 2 commits into
aetherpak:mainfrom
TingPing:fix/records-cell-branch
Open

TingPing wants to merge 2 commits into
aetherpak:mainfrom
TingPing:fix/records-cell-branch

Conversation

@TingPing

@TingPing TingPing commented Sep 7, 2026

Copy link
Copy Markdown

Problem

Pushing two branches of the same app and arch in one execution loses one of them from the published index.

CellDir builds the record path from the app ID and arch only:

return filepath.Join(root, fmt.Sprintf("%s-%s", r.AppID, r.Arch)), nil

so both pushes write into <records-dir>/<app-id>-<arch> and the second record.json overwrites the first. build-site merges from those cells, so only the last branch pushed reaches the static index.

Observed while publishing a master and a 2.54 branch of two apps in one run — five pushes, three cells:

Record wrote to cell: _records/org.example.app-aarch64
Record wrote to cell: _records/org.example.app-x86_64        <- 2.54
Record wrote to cell: _records/org.example.app-x86_64        <- master, overwrites
...
Found 3 execution records to merge.

The failure is quiet and quite confusing from the outside: both branches are exported, signed, and pushed to the registry under correct, distinct tags (…-2.54-x86_64 and …-master-x86_64), and every step reports success — but only -master entries appear in index/static and only -master.flatpakref files get generated, so the other branch cannot be installed even though its images are present.

Fix

Put the branch in the cell path, matching the OCI tag in pkg/oci/oci.go, which already distinguishes branches as <app-id>-<branch>-<arch>.

Two existing properties keep the change contained:

  • IterRecords locates cells by walking for record.json/labels.json and reads the branch from the JSON, so it never parsed the directory name.
  • mergeRecord keys index images by org.flatpak.ref + architecture, and the ref already carries the branch, so nothing downstream re-collapses the two.

A record with no branch keeps the old path, so callers that never set one are unaffected and previously written cells still load.

Since the branch becomes a path segment, Validate now checks it against branchRegexp when set — previously a branch like ../escaped was unvalidated while app ID and arch were.

Testing

  • make fmt, make vet (also with -tags=integration) and make test are clean.

  • New TestWriteRecordSeparatesBranches covers two branches of one app and arch. With the CellDir change reverted it fails, so it pins the regression:

    expected distinct cells for two branches of one app and arch,
    got ".../org.example.App-x86_64" for both
    
  • Added a Validate case for a path-traversal branch.

  • Updated the record paths asserted in tests/push_autodetect_test.go to the new layout, derived from each scenario's mock branch. Scenario 3a's single App1-x86_64 assertion previously covered both x86_64 branches, so its beta cell is now asserted explicitly rather than losing that coverage.

  • ARCHITECTURE.md documented the old layout and now describes both it and the fallback.


Second problem: a dotted branch makes the image unsignable

Once both branches reach the index, installing the one whose branch contains a dot fails:

Identity in signature (ghcr.io/org/repo:org_example_App-2.54-x86_64)
  does not match ghcr.io/org/repo
error: Failed to install org.example.App: no valid signatures for ghcr.io/org/repo@sha256:…

The tag is built as <app-id>-<branch>-<arch> with dots stripped from the app ID, and the comment there says why — signature tag strip mismatches — but the branch is interpolated raw:

safeAppID := strings.ReplaceAll(opts.AppID, ".", "_")
tag := fmt.Sprintf("%s-%s-%s", safeAppID, opts.Branch, opts.Arch)

flatpak strips the tag off a signature's docker-reference with ^(.*?)(?::TAG)?(?:@DIGEST)?$, where its TAG pattern is [0-9A-Za-z_][0-9A-Za-z_-]{0,127} — no dot, though OCI tags allow one. A dotted tag therefore can't match the optional tag group, the lazy first group swallows the whole reference, and the identity check compares the full registry/repo:tag against registry/repo:

tag stripped identity
org_example_App-master-x86_64 ghcr.io/org/repo
org_example_App-2.54-x86_64 ghcr.io/org/repo:org_example_App-2.54-x86_64

So the existing dot handling was right, just not applied widely enough. CleanTag already exists for this and covers the app ID too, so the tag is now built through it, giving org_example_App-2_54-x86_64. Tags without dots — every current one — are byte-identical to before.

TestTagHasNoDots covers the three shapes, including a dotted app ID with a dotted branch.

Fixing it here rather than in flatpak keeps published repositories installable with released flatpak versions.

Publishing two branches of one app and arch in a single execution had
both pushes write into <records-dir>/<app-id>-<arch>, so the second
record overwrote the first. build-site then merged only the surviving
branch: the other branch's images were pushed to the registry and
tagged correctly, but never appeared in the static index or as a
.flatpakref, leaving them impossible to install.

The OCI tag already distinguishes branches, so let the cell path do the
same. Records written without a branch keep the old path, and IterRecords
reads the branch from record.json rather than the directory name, so
existing cells still load.
The app ID was stripped of dots to avoid signature tag strip mismatches,
but the branch was interpolated raw, so a branch like 2.54 produced a tag
flatpak cannot parse: its docker-reference tag pattern omits '.', so
stripping the tag off the signature identity leaves it intact and the
image is rejected as unsigned.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant