Write a language-agnostic protocol specification (PROTOCOL.md) #367

Activity

1. beardthelion

   added

   kind:docsDocs and comments onlyDocs and comments only

   sev:lowCosmetic, cleanup, or nice-to-haveCosmetic, cleanup, or nice-to-have

   subsystem:apiNode REST API request/response surfaceNode REST API request/response surface

   subsystem:attestationCertificates, anchoring, per-ref attestationCertificates, anchoring, per-ref attestation

   subsystem:identityDID/UCAN, http-sig auth, push authorizationDID/UCAN, http-sig auth, push authorization

   subsystem:replicationMirror, replica, and cross-node syncMirror, replica, and cross-node sync

   on Aug 18, 2026
2. beardthelion commented on Sep 9, 2026

   

   beardthelion

   on Sep 9, 2026

   Collaborator

   More actions

   @andreolf you asked here for scope and structure before writing, and nobody answered. That is on me,
   and it went worse than a slow reply: #372 opened doing this work, and what discussion there was
   happened there instead of here, so you had no way to see it from this issue.

   Where it actually stands, stated as it is rather than as a decision: kevincodex1 raised the
   possibility on #372 of putting the protocol and specs in a separate repo to keep them clean, and said
   he would ping when it is ready. Nothing has been created yet, and there is no decision recorded
   anywhere else. So the location question you asked is still genuinely open, not settled behind your
   back.

   On the normative/informational split: yes, and the code argues for a third category you did not ask
   for. There are places where the implementation is internally consistent, documented, and deliberately
   NOT what the relevant RFC says. Those need marking as "true of this implementation, non-standard, do
   not follow the RFC here", because an implementer who trusts the RFC over the spec builds something
   that fails in a way the spec would otherwise be blamed for. The clearest one: @path in our RFC 9421
   signing string carries the query, bound from path_and_query(), where 9421 defines @path as
   path-only with @query separate. It bites immediately, because ?service= is mandatory on the one
   signed GET in the git transport.

   Where a wrong sentence actually causes silent failure

   I had assumed sections 2 and 4 were the hazard. Section 2 is. Section 4 is not, and the reason is
   worth knowing before you spend time on it.

   The real landmine is canonicalization, and it cuts across both. There are three different schemes in
   the tree and two of them are labelled canonical:

   • RefUpdateBody::to_signing_bytes is documented "canonical JSON bytes for signing" but is
     serde_json::to_vec over a struct, so it is declaration order.
   • gitlawb-attest::cert_hash is genuine JCS (RFC 8785).
   • gitlawb-node's own issue_ref_certificate signs a serde_json::json! Value, and with no
     preserve_order feature that Map is a BTreeMap, so those bytes are lexicographic.

   serde_jcs appears only in gitlawb-attest. So an implementer who reads that crate's excellent "JCS
   so they reproduce across implementations" line and applies it to the certificate's own signature
   produces bytes that never verify, with no diagnostic beyond a signature failure. If the spec says one
   sentence about canonicalization it has to say which of the three applies where.

   Section 4 is a different problem: the ref-update certificate has no wire presence at all.
   RefUpdateCert appears in no node, client, or remote-helper code; gitlawb-attest is a workspace
   member nothing depends on. It is also unconstructible as written, requiring 64-hex object ids while
   the node creates repos with --object-format=sha1 and parse_ref_updates hard-codes 40. I would
   mark it planned rather than specify it, and say so explicitly, since a spec that documents an unused
   type as protocol is worse than one that omits it.

   The thing in that area that IS load-bearing and unwritten: a SHA-256 repo's push parses to zero ref
   updates because of that same 40-char assumption, so branch protection, certificates, webhooks and
   replication all silently no-op while the push itself succeeds.

   Section 8, and a correction to your framing

   Your instinct to split normative from informational matters most here, and the honest version is
   blunter than yours: essentially all of it is optional, and two of its three claims do not describe
   shipped behavior. IPNS is not implemented, one comment mentions it; branch tracking is a SQL upsert,
   unsigned, last-write-wins. And the git-to-CID mapping is not a SHA-256 correspondence: the CID digests
   the unframed payload while git digests the framed object, so they are different by construction, and
   the reverse lookup goes through a table rather than arithmetic. /ipfs/{cid} is not a network read at
   all; it serves from the local git object store.

   What your ten sections miss

   This is the part worth your time, and it is why the scope conversation was worth having before
   writing rather than after. In rough order of how badly a second implementation breaks without it:

   1. UCAN. X-Ucan is read and fully validated on every signed request, and its encoding is not the
      standard UCAN JWT: it is {payload, s} signed over declaration-order JSON. Someone who knows UCAN
      builds a JWT and gets 401. It currently grants no authority, which is itself a thing to state.
   2. GraphQL. An entire second API surface, POST /graphql plus /graphql/ws subscriptions, with
      its own queries, mutations and subscriptions, mounted outside the auth layer for the socket.
   3. Path-scoped visibility. Reads are gated per path, and the packfile itself is filtered, so a
      clone of such a repo yields a reachability-incomplete pack that a stock full clone rejects. There
      is a withheld-paths endpoint that exists precisely so a client can sparse-exclude. Normative for
      any second client, absent from your list.
   4. Repo addressing and DID normalization. {owner} is a DID, not a username, and the path segment
      is the bare multibase key with did:key: stripped, case-sensitively. Nothing in the scope says how
      a client turns a DID into a URL.
   5. The error envelope and status taxonomy, including the non-obvious ones: 429 with retry-after,
      503 with Retry-After for capacity shedding, 504 for git timeouts, and a 413 whose body is plain
      text rather than the JSON envelope every other error uses.
   6. Custom headers, notably WWW-Authenticate: Signature realm="gitlawb-alpha", alg="ed25519" on
      unauthenticated push. There is no Basic challenge, so git credential helpers are inert.
   7. Smart-HTTP specifics you have as two endpoints: ?service= mandatory with no dumb-HTTP
      fallback, protocol v0 only with GIT_PROTOCOL never propagated, and no gzip request decompression
      anywhere, so a gzipped request from stock git is a 400.
   8. Webhooks, which are a wire contract with a different signature scheme entirely: HMAC-SHA256 in
      X-Gitlawb-Signature-256, not Ed25519, not 9421.
   9. The encrypted blob envelope: GLENC magic, version 2, XChaCha20-Poly1305 with per-recipient
      X25519-from-Ed25519 key wrapping, served over its own routes.
   10. Behavior that varies by operator config and therefore cannot be assumed by a client, including
       owner-push enforcement, signed-peer writes, and the iCaptcha mode, which defaults to off.
   11. Version discriminants, plural. You name one. There are at least seven distinct versioned
       formats in the tree.

   Two smaller corrections while you are in here. Identity: did:web and did:gitlawb parse and
   validate, but only did:key resolves to a verifying key, and anything else is a 400 at auth, so the
   spec should scope identity to did:key and say the others are reserved. iCaptcha: the node reads only
   x-icaptcha-proof from a request; x-icaptcha-url and x-icaptcha-level are response-only.

   The model to copy

   crates/gitlawb-attest is already the thing you are proposing, for one narrow surface, and it is
   good: wire shape, JCS canonical bytes, domain-separated signing input, and a canonical-vectors test
   whose own docstring says it exists so a port to another language has an oracle to check against
   without reading Rust. Whatever shape the spec takes, that plus vectors is the bar. A spec section
   without a test vector is a section that will drift.

   What #372 empirically taught

   Worth knowing since it is now evidence rather than prediction. The first review round was ordinary
   factual accuracy. The second was the harder class: statements individually true that still lead an
   implementer to build something broken. That is the argument for your split, plus the third category
   above.

   If you still want to drive this, say so here and I will make sure you are looped in wherever it lands
   rather than finding out through a PR again. Your storage docs in #363 and the verification work in
   #365 and #366 are the right background for it, and the canonicalization question above is the one I
   would most want your eyes on regardless of who writes the document.