Skip to content

About

Where personal data goes: logs, third-party SDKs, LLM providers and outbound HTTP. Offline, deterministic static analysis with cited paths, SARIF and Fides.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

1 star

Watchers

0 watching

Forks

Latest commit

 

History

88 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

privacy-flow logo

privacy-flow

Deterministic, offline static analysis of where personal data goes: logs, third-party SDKs, LLM providers and outbound HTTP. Cited file:line flow paths, SARIF, canonical JSON and Fides egress.

Where does the email address go?

A record of processing needs to know which processors receive personal data, and most GDPR findings in code review are flows, not columns: a request body logged in full, an email address passed to an analytics SDK, a customer record pasted into an LLM prompt. piiflow finds those flows and cites every hop. Offline, reproducible, no model.

release ci OpenSSF Scorecard License: MIT crates.io DOI

Install

The binary is called piiflow; the package is privacy-flow everywhere.

Homebrew

brew install noru-tech/tap/piiflow        # macOS and Linux

crates.io

The crate is privacy-flow (the short names pflow on crates.io and on PyPI belong to unrelated tools). The binary it installs is piiflow.

cargo binstall privacy-flow               # prebuilt binary from GitHub Releases
cargo install privacy-flow --locked

Prebuilt binaries

Download an archive for your platform from GitHub Releases: fully static Linux builds (x86_64-unknown-linux-musl, aarch64-unknown-linux-musl) and macOS builds (aarch64-apple-darwin, x86_64-apple-darwin), each named privacy-flow-<target>.tar.xz and holding piiflow, the README, the changelog, the licence and the notice.

Verify before you run

Every archive carries a GitHub artifact attestation from the release workflow and a SHA-256 checksum file next to it (<archive>.sha256; sha256.sum lists all of them). Check both before you unpack:

VERSION=v0.5.1
ARCHIVE=privacy-flow-aarch64-apple-darwin.tar.xz
gh release download "$VERSION" --repo noru-tech/privacy-flow \
  --pattern "$ARCHIVE" --pattern "$ARCHIVE.sha256"

# Built by this repository's release workflow, from a tagged commit
gh attestation verify "$ARCHIVE" --repo noru-tech/privacy-flow \
  --signer-workflow noru-tech/privacy-flow/.github/workflows/release.yml

# The bytes match the published checksum (on Linux: sha256sum -c "$ARCHIVE.sha256")
shasum -a 256 -c "$ARCHIVE.sha256"

tar -xJf "$ARCHIVE"

There is no curl | sh installer in these instructions, and there will not be one.

Quick start

Try it on the demo in this repository, a sign-up function that logs, tracks and prompts:

$ piiflow scan examples/demo -f table
PF001  medium   pf-f9eecf0af7d241bc  src/users.ts:14:3
                user.contact.email from `user.email (user: User)` (src/users.ts:12) reaches a log sink (js.pino) at src/users.ts:14
PF002  high     pf-fc2d508c092dd964  src/users.ts:15:3
                user.contact.email, user.unique_id.pseudonymous from `user.id (user: User)` (src/users.ts:12) and 1 other source reaches PostHog, a processor not declared in .privacy-flow.yml, at src/users.ts:15
PF002  high     pf-b636117398064a06  src/users.ts:17:9
                user.contact.phone_number from `user.phone_number (user: User)` (src/users.ts:12) reaches OpenAI, a processor not declared in .privacy-flow.yml, at src/users.ts:17
PF003  medium   pf-309131a80c57fa82  src/users.ts:17:9
                user.contact.phone_number from `user.phone_number (user: User)` (src/users.ts:12) reaches OpenAI through a model call at src/users.ts:17

5 flow(s) from 3 source(s) to 4 sink(s); findings: PF001 1, PF002 2, PF003 1
processors reached: OpenAI, PostHog
coverage: complete

Scan a repository, see why, and enforce a policy:

piiflow scan .                            # writes .privacy-flow/flows.json
piiflow explain pf-0123456789abcdef       # every hop, with the source line of each
piiflow check .privacy-flow/flows.json    # exit 1 above the threshold, 4 with open coverage gaps

Or report the flows each pull request introduces, as code-scanning alerts, with the GitHub Action:

# .github/workflows/privacy-flow.yml
on: pull_request
permissions:
  contents: read
  security-events: write
jobs:
  privacy-flow:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
        with: { fetch-depth: 0 }
      - uses: noru-tech/[email protected]

How do I find where personal data goes in my code?

Trace each value that holds personal data to every place it leaves the program, and cite the chain: piiflow does that for TypeScript, JavaScript, Python and C#, offline and deterministically, and reports each path as a chain of file:line:column hops from the source to the sink.

$ piiflow explain pf-309131a80c57fa82
pf-309131a80c57fa82 PF003 medium: user.contact.phone_number from `user.phone_number (user: User)` (src/users.ts:12) reaches OpenAI through a model call at src/users.ts:17
flow-1b97b0cc5aa9541b (user.contact.phone_number, call depth 2):
   1. source  src/users.ts:12:30
      user.contact.phone_number (typed_object: User.phone_number, field declared at src/users.ts:10; classified by classification-table)
         12 | export async function signup(user: User) {
   2. call    src/users.ts:16:18
      into format (parameter u)
         16 |   const prompt = format(user);
   3. read    src/format.ts:2:18
          2 |   return `Call ${u.phone_number} about ${u.plan}`;
   …
  11. sink    src/users.ts:17:9
      llm sink js.openai, processor OpenAI
         17 |   await openai.chat.completions.create({ model: 'gpt-4o', messages: [{ role: 'user', content: prompt }] });

The helper format does not use the email address, so the email address does not reach the model: the flow through the helper is computed per argument and per field.

  • Sources are reads of fields the classification table names (user.email, customer.phone_number), parameters named that way, objects typed with a type that declares such fields, request input in Express, Fastify, Next.js, FastAPI, Flask, Django, ASP.NET Core (minimal APIs, MVC controllers, gRPC services, claims) and Blazor (bound form values, supplied parameters), and fields from an existing Fides data map. Categories are Fideslang keys.
  • Sinks are logs (console, pino, winston, Python logging, structlog, ILogger and [LoggerMessage], Serilog …), error tracking (Sentry, Application Insights), product analytics (Segment, Mixpanel, PostHog, Amplitude), email, SMS, chat, push and message brokers (SMTP, Nodemailer, Django and Flask mail, SendGrid, Postmark, Twilio, Slack, Firebase, RabbitMQ, Azure Service Bus, Kafka …), LLM providers (OpenAI, Anthropic, Gemini, Mistral, LangChain, the Vercel AI SDK, Microsoft.Extensions.AI), outbound HTTP (fetch, axios, requests, httpx, HttpClient …, with the host when it is a literal), browser storage, and other processors (Stripe, S3 …). Each names the processor that receives the data.
  • The analysis is field-sensitive (user.id does not carry user.email), follows calls into the project's own functions through summaries (a helper called with an email address and with an ID does not mix them), across files, packages, tsconfig aliases and C# namespaces (with global and implicit usings, partial classes, extension methods and MediatR dispatch), and is flow-insensitive within a function.
  • Unknown is reported as unknown. A call the analysis cannot place, a framework it does not model, a file it could not parse: each is a coverage gap, and a result with open gaps never exits as clean.

What is in this repository

Piece Where
CLI — piiflow: scan, diff, explain, check, validate, rules, doctor src/
Catalogue — sources, sinks, sanitisers, propagators, frameworks, as YAML catalogue/, docs
Schemas — flow-facts document (stable, versioned) and configuration (JSON Schema 2020-12) schemas/
Rule pages — one per rule, with controls, examples, fixes and dispositions docs/rules/
Outputs — table, canonical JSON, SARIF 2.1.0 with code flows, Fides egress, in-toto, facts docs
Data map — piiflow datamap: a full Fides data map (systems, datasets, flows, proposed declarations), merged in place docs
GitHub Action — the flows a pull request introduces, as SARIF and a job summary action.yml, docs
Conformance corpus — programs with expected flows, and a runner any analyser can use conformance/
Design — how it works, and an ADR per major choice design, ADRs
Benchmark — piiflow 0.1.0 against Privado on real applications, labelled blind by two reviewers: on the ten labelled, 72 % of the flows piiflow reports are real (Privado 22 %), and it finds 32 % of sampled sinks that receive personal data (Privado 17 %), 40 % counting the places it flags; piiflow 0.2.0 finds 55 % of the same sinks (its precision is not measured yet) report · scores · method

How fast is it?

On the same machine, piiflow scanned each application in 0.11 s to 40.8 s, using at most 624 MiB of memory. Privado took 10.4 s to 14 min and up to 10.6 GiB, and had not finished Polar after 60 min. Half the applications ran more than 94 times faster with piiflow.

Time to scan each application, piiflow and Privado

Application Lines of code piiflow Privado piiflow memory Privado memory
FastAPI template 1,520 0.11 s 10.4 s 14 MiB 1.3 GiB
Taxonomy 7,788 0.20 s 20.5 s 22 MiB 759 MiB
Open SaaS 10,774 0.20 s 23.1 s 23 MiB 1.3 GiB
Healthchecks 17,110 0.57 s 51.2 s 30 MiB 2.8 GiB
Vercel chatbot 20,446 0.27 s 23.0 s 32 MiB 2.3 GiB
CTFd 23,488 0.63 s 41.0 s 35 MiB 3.7 GiB
Hoppscotch 27,376 0.71 s 48.6 s 47 MiB 2.4 GiB
Redash 27,804 0.91 s 7.7 min 49 MiB 10.6 GiB
PrivateGPT 96,645 4.43 s 89.1 s 133 MiB 4.6 GiB
Umami 107,365 2.54 s 4 min 152 MiB 4.5 GiB
Ghost 160,029 3.95 s 14 min 192 MiB 6.3 GiB
Polar 222,747 40.8 s not finished after 60 min 624 MiB 12.8 GiB

How this was measured (2026-10-02). The 12 open-source applications of the benchmark corpus; each one on its own GitHub-hosted runner (4 vCPU, 15.6 GiB; 4 different CPU models across jobs, listed in results.json), with both tools run one after the other on it. piiflow 0.1.0 is the released Linux binary, verified by its attestation; the median of 3 runs is shown, and its output matched the recorded macOS run byte for byte on every application. Privado is privado-core 1.1.175 from its pinned image with its newest rules, offline, one run, 14 GiB limit. This measures speed and memory only; whether each tool's findings are right is measured by the labelled benchmark. Method and how to rerun it.

What does piiflow not do?

It does not find security vulnerabilities, scan data at rest or in databases, observe the running system, or decide purposes, legal bases or data subjects: those stay human (or Noru) judgements. The data map proposes data uses from the code's evidence, each marked for review, and never a legal basis. It never calls a model, makes no network calls, and sends no telemetry. A clean result is a statement about the scanned scope, with its gaps listed, not a compliance certification.

The analysis is flow-insensitive within a function, follows plain objects one field deep, and supports TypeScript, JavaScript, Python and C# with the frameworks listed above. Everything it does not cover is in KNOWN-LIMITATIONS.md; what comes next is in the roadmap.

Which rules does piiflow check?

Each rule has a page that explains it, maps it to controls, shows a failing and a passing example and says how to fix it or record a disposition (all rules):

ID Finding Default severity
PF001 Personal data reaches a log sink medium
PF002 Personal data reaches a third-party processor not declared in config high
PF003 Personal data reaches an LLM provider medium
PF004 Special-category data (Art. 9 or 10) reaches an external sink high
PF005 Credentials or authentication data reach a log or third party high
PF006 Personal data reaches outbound HTTP with a non-literal host warning
PFC01 Coverage gap: unsupported framework, dynamic dispatch or depth bound reached warning

Declaring a processor in .privacy-flow.yml (with a citation to the DPA or vendor record) turns PF002 into an informational egress fact: that is how the tool fits a team that already keeps a processor list. The control mapping is pending review by a privacy lawyer or Noru's compliance lead.

Commands

Command What it does
piiflow scan [PATH] Analyse and write findings and flow facts (default .privacy-flow/flows.json)
piiflow diff BASE..HEAD [PATH] Report only flows introduced or changed between two revisions (BASE.. compares with the working tree)
piiflow datamap [PATH] Write or update the repository's Fides data map (default .fides/datamap.yml): systems, datasets, egress and ingress, proposed declarations; --check exits 5 when it is out of date
piiflow explain ID Print a finding's (or flow's, or gap's) hop chain with the source line of each hop
piiflow check FILE Enforce policy and dispositions (--as-of DATE, --fail-on); exit 1 on failure
piiflow validate FILE Re-check a document or in-toto Statement: schema, digest, derived findings
piiflow rules List the loaded catalogue, its version and every source, sink, sanitiser and propagator
piiflow doctor [PATH] Show detected languages, frameworks, configuration and coverage before a scan
piiflow completions SHELL / piiflow manpage DIR Shell completions and man pages

Global flags: -q silences status lines; --threads N sets worker threads (the output is the same for every value).

Output formats and exit codes

--format table|json|sarif|fides|in-toto|facts and --output FILE; the format is inferred from the file name when omitted (.sarif, .json, .yml, .intoto.json, .jsonl). JSON is RFC 8785 canonical bytes and carries a SHA-256 digest that validate recomputes (output).

Code Meaning
0 Success, or the policy threshold passed
1 Policy threshold exceeded (check)
2 Invalid command-line arguments
3 Invalid input: configuration or document
4 Coverage incomplete: a clean result cannot be claimed (takes precedence over 1)

What each code means and what to do about it: exit codes.

Configuration

.privacy-flow.yml declares processors, extra sources, sinks and sanitisers (each with a citation), field names, names that are not personal, exclusions and policy, validated against a published schema. See configuration and examples/.privacy-flow.yml.

Conformance

The conformance corpus is a set of small programs with hand-written expected flows and an external-runner contract (<cmd> <vector-dir>, flows as JSON lines on stdout), so any analyser can be scored against it:

python3 conformance/run.py --verifier "piiflow -q scan --walk -f facts"

Development

cargo build
cargo test            # unit, CLI, fixtures, conformance, determinism goldens, principles
cargo clippy --all-targets -- -D warnings
cargo bench           # criterion benchmarks, including the engine comparison

See CONTRIBUTING.md for the layout, how to add an SDK to the catalogue, and how goldens are updated.

How to cite

Every release from 0.1.1 on is archived on Zenodo. Cite the concept DOI 10.5281/zenodo.23104908, which resolves to the latest release, or the version DOI of the release you used, listed on that record, when the exact bytes matter (an audit report, a benchmark, or a conformance claim). CITATION.cff has the citation metadata.

Trust

How a release gets from this repository to your machine, and how to check it:

  • Built in CI, from a tag. Releases are built by cargo-dist in GitHub Actions (release.yml) from the tagged commit.
  • Attested. Every archive has a GitHub artifact attestation (SLSA build provenance, signed through Sigstore) naming the workflow and commit that built it; see Verify before you run.
  • Checksums and SBOM. Each archive has a .sha256 file, sha256.sum lists them all, and a CycloneDX SBOM is attached to every release.
  • Offline by construction. No network crate is a dependency, and a test fails if one is added or if the source opens a socket or starts a process other than git.
  • Pinned and scored. Every GitHub Action is pinned to a commit SHA and kept current by Dependabot; supply-chain posture is measured by OpenSSF Scorecard.
  • Vulnerabilities. Report privately through GitHub private vulnerability reporting; see SECURITY.md.

Relationship to Noru

piiflow is standalone and needs no Noru account. Parsing, the intermediate representation, the flow analysis, the catalogue, the rules, every output format and the data map (with its proposals, for review) are open, here; deciding purposes and legal bases, record-of-processing assembly and cross-repository topology are Noru's (NOTES.md records the line). privacy-datamap can read the flow facts to fill processor and egress fields in a data map, and ai-inventory can read PF003 facts to answer "what data reaches the model"; the flow-facts schema is stable and versioned for them.

License

MIT, see LICENSE. The tool, the catalogue, the schemas and the conformance corpus are MIT-licensed and independent of the Noru platform. The vendored Fideslang taxonomy is © Ethyca, Inc. under CC BY 4.0; see NOTICE.

Maintained by Noru, a continuous compliance platform.

About

Where personal data goes: logs, third-party SDKs, LLM providers and outbound HTTP. Offline, deterministic static analysis with cited paths, SARIF and Fides.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages