Skip to content
ADITYASINGH1206Public

About

Calibrated & auditable video deepfake screening service with spatiotemporal patch heatmaps, cryptographic Ed25519 detection receipts, EVM Merkle anchor, and reviewer console

Resources

Contributing

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

VeriFrame: Calibrated, Auditable Video Deepfake Screening

Formerly SENTINEL-V (see docs/DECISIONS.md, ADR 007). Internal package names (sv_*), SV_ settings and the sv-receipt/1 format are unchanged.

VeriFrame is a video deepfake screening platform and reviewer console built on the S-MSDT detector family. It provides prevalence-calibrated probabilities, explainable patch-evidence heat-maps, and cryptographically signed detection receipts anchored on EVM blockchains.

       ┌────────────────┐
       │ Video Upload   │
       └───────┬────────┘
               │
               ▼
   ┌───────────────────────┐
   │ Ingest & Preprocess   │ ── Detect & Track Faces (Buffalo_L)
   └───────────┬───────────┘
               │
               ▼
   ┌───────────────────────┐
   │ Cost-Aware Cascade    │ ── S-MSDT-S (T=1) ── Exit Early if Authentic
   └───────────┬───────────┘          │ Escalated
               ▼                      ▼
   ┌───────────────────────┐   S-MSDT-ST (T=8)
   │ Evidence & Fusion     │ ── Warped Patch Maps + C2PA + Generative Signals
   └───────────┬───────────┘
               │
               ▼
   ┌───────────────────────┐
   │ Signed Receipt        │ ── RFC 8785 Canonical JSON + Ed25519
   └───────────┬───────────┘
               ▼
   ┌───────────────────────┐
   │ EVM Merkle Anchor     │ ── Immutable Audit Trail
   └───────────────────────┘

Key Features

  1. Prevalence-Calibrated Screening: Probability estimation calibrated against real-world base rates ($\pi$) rather than arbitrary binary thresholds.
  2. Two-Stage Cost Cascade: Fast screening via S-MSDT-S ($T=1$), escalating uncertain or suspicious clips to S-MSDT-ST ($T=8$); the compute saving is measured in experiment E3 rather than assumed.
  3. Forensic Evidence Maps: Spatiotemporal patch heat-maps warped back to source video coordinates for frame-by-frame inspection in the Reviewer Console.
  4. Verifiable Detection Receipts: Cryptographic audit trail linking media SHA-256, model weights hash, calibration hash, and decision policy, anchored to an EVM testnet.
  5. Non-Invasive SENTINEL Integration: Additive compatibility layer for the SENTINEL Web3 content-verification platform.

Monorepo Layout

├── AGENTS.md                  # Autonomous agent instructions & rules
├── docs/                      # Specification, ADRs, blockers, architecture, and developer guide
│   ├── SPEC.md                # Authoritative Build Specification v1.0
│   ├── SENTINEL-V_Build_Spec.md # Verbatim build specification reference
│   ├── ARCHITECTURE.md        # Deep subsystem architecture & data flows
│   ├── CONTRIBUTING.md        # Contributor workflow & parallel stream rules
│   ├── DECISIONS.md           # Architecture Decision Records (ADRs)
│   └── QUESTIONS.md           # Blocker tracking log (B1–B8)
├── skills/                    # Specialized agent & developer skills
│   ├── parity-verification/   # S-MSDT numerical equivalence checks (§6.9)
│   ├── model-bundle-builder/  # Immutable bundle builder & safetensors converter
│   ├── paper-b-evaluation/    # Paper B evaluation harness & experiment runners
│   ├── receipts-and-anchoring/# RFC 8785 JCS, Ed25519 signing & EVM anchoring
│   ├── sentinel-integration/  # Additive SENTINEL integration & webhook signing
│   └── reviewer-console-engineering/ # React console, canvas overlay & keyboard UX
├── packages/
│   ├── sv_core/               # Shared settings, schemas, errors, canonical JSON
│   ├── sv_engine/             # Preprocessing, S-MSDT adapters, calibration, evidence
│   ├── sv_fusion/             # C2PA provenance and generative image signals
│   └── sv_receipts/           # Receipt generation, Ed25519 signing, Merkle trees
├── services/
│   ├── api/                   # FastAPI gateway (analyses, receipts, review queue)
│   ├── workers/               # Celery workers (ingest, preprocess, gpu, finalize, anchor)
│   └── monitor/               # Drift detection (PSI), rolling ECE, retention
├── web/                       # Reviewer Console SPA (React, TypeScript, Vite, Tailwind)
├── contracts/                 # Foundry project with ReceiptAnchor.sol
├── integrations/sentinel/     # Additive adapter for SENTINEL platform
├── models/                    # Model bundle definitions and model cards
├── eval/                      # Paper B experimental evaluation harness
├── tools/                     # Calibration fitting, cascade tuning, parity checks
└── infra/                     # Dockerfiles, docker compose, Prometheus, Grafana

Quick Start

Prerequisites

  • Python 3.11+
  • Node.js 20+
  • Docker & Docker Compose
  • NVIDIA GPU with CUDA 12.8 (optional; mock mode runs CPU-only)

Development (Mock Engine Mode)

# Clone and enter repository
git clone https://github.com/ADITYASINGH1206/veriframe.git
cd veriframe

# Initialize development environment
make dev

Visit the services:

Web app only (no backend)

cd web
npm ci
npm run dev            # http://localhost:5173 — falls back to labelled DEMO DATA when the API is not running
VITE_DATA_MODE=demo npm run dev   # force generated demo data everywhere

The console marks every generated item with a DEMO DATA badge. The synthetic demo video in web/public/demo/ is produced by web/scripts/make_demo_video.py and contains no real person.

Running Tests

# Unit and integration tests (mock mode)
make test

# Code quality checks
make lint

Parallel Development Streams

VeriFrame is built concurrently across two parallel workstreams:

  • Stream A — ML & Data Forensics: Lead by @sharadpolackalsunil

    • Video preprocessing, face tracking, and clip formation (M1)
    • S-MSDT adapters, checkpoint verification, and calibration tooling (M2)
    • Patch evidence generation and video heat-map overlays (M4)
    • Cascade exit tuning and policy math (M6)
    • Drift monitoring (PSI) and rolling ECE (M9)
    • Evaluation harness and Paper B experiments (M10)
  • Stream B — Systems, Full-Stack & Web3: Lead by @AdityaRaj1304

    • Monorepo tooling, Docker Compose, and CI/CD pipelines (M0)
    • FastAPI gateway, Celery task pipelines, and Redis pub/sub (M3)
    • Reviewer Console web application and keyboard workspace (M5)
    • Detection receipts, Ed25519 signing, and EVM contract anchoring (M7)
    • SENTINEL platform integration adapter (M8)
    • Security hardening, SSRF sandboxing, and load testing (M11)

See GitHub Issues for detailed task specifications and acceptance criteria.


Agent & Developer Skills

The repository includes curated, executable skill modules under skills/ designed to provide deep procedural knowledge for both coding agents and human engineers:

Skill Path Description & Scope
Parity Verification skills/parity-verification/ Step-by-step verification protocol for GPU inference parity ($
Model Bundle Builder skills/model-bundle-builder/ Safetensors conversion, offline CLIP backbone export, Platt calibration fitting, and checksum generation.
Paper B Evaluation skills/paper-b-evaluation/ Reproducible execution of experiments E1–E12, dataset manifests, and publication figure plotting.
Receipts & EVM Anchoring skills/receipts-and-anchoring/ RFC 8785 canonical JSON, Ed25519 signatures, Merkle trees, and Foundry ReceiptAnchor.sol anchoring.
SENTINEL Integration skills/sentinel-integration/ Purely additive integration adapter, webhook HMAC validation, and locked-file preservation.
Reviewer Console Engineering skills/reviewer-console-engineering/ Canvas heat-map synchronization via requestVideoFrameCallback, keyboard shortcuts, and WCAG 2.2 AA.

About

Calibrated & auditable video deepfake screening service with spatiotemporal patch heatmaps, cryptographic Ed25519 detection receipts, EVM Merkle anchor, and reviewer console

Resources

Contributing

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages