Skip to content

Verify fetched IPFS content against its CID via CAR - #202

Open
cyc60 wants to merge 6 commits into
masterfrom
ipfs-hash-verification
Open

Verify fetched IPFS content against its CID via CAR#202
cyc60 wants to merge 6 commits into
masterfrom
ipfs-hash-verification

Conversation

@cyc60

@cyc60 cyc60 commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Description

IpfsFetchClient previously fetched {gateway}/ipfs/{cid} and returned the bytes without checking them against the CID. Consumers treat the CID as an on-chain commitment (Keeper config/rewards IPFS hashes, GENESIS_VALIDATORS_IPFS_HASH), so unverified content is an integrity gap: a malicious or MITM'd gateway could serve arbitrary bytes for a committed CID.

Fetched content is now verified against its CID by requesting the trustless-gateway CAR response (?format=car) and decoding it with ipfs-car-decoder, which re-hashes every block against the requested CID. This covers raw and dag-pb codecs, CIDv0/v1, and single- and multi-block DAGs, so it works for config, rewards, and the large genesis-validators lists alike.

Behavior with verify_hash=True (default):

  • Gateway serves a CAR that decodes → verified bytes returned.
  • Gateway serves a CAR that fails to decode (block mismatch / corrupt / truncated) → fail closed; never falls back.
  • No gateway serves a CAR and no mismatch was detected → log an error and fall back to a plain unverified fetch (availability over strict integrity; the failure is classified by response Content-Type).
  • Local IPFS node cat → trusted (content-addressed).
  • S3 endpoints → lightweight raw-CIDv1 sha256 check (oracle's S3 payloads), fail closed on any other CID.

fetch_json now parses verified bytes. verify_hash=False keeps the previous plain-fetch behavior as an escape hatch.

Dependencies: replaced py-cid / py-multihash with ipfs-car-decoder and multiformats, pinned exactly because verification relies on the decoder's validate=True block-checking.

Note for downstream: a custom gateway configured by an operator must support trustless CAR responses (?format=car); all shipped default gateways (Filebase, Quicknode, Pinata) do.

@cyc60
cyc60 force-pushed the ipfs-hash-verification branch from 9f219ac to dc85f74 Compare September 3, 2026 08:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant