A self-hosted workspace built on the compression ideas from Headroom and the workspace design of Odysseus.
AI tools send a lot of text to the model. Most of it is repetitive — file listings, build logs, JSON blobs. You pay for every character of it.
Sensei sits in the middle and squeezes that text down before it's sent. The model still gets everything it needs; you just stop paying for the padding.
flowchart LR
A["🧑💻 Your AI tool<br/><sub>Claude Code, Cursor, Aider…</sub>"]
B["🥋 Sensei<br/><sub>runs on your machine</sub>"]
C["☁️ The model<br/><sub>Anthropic, OpenAI, or local</sub>"]
A -- "50,000 tokens" --> B
B -- "10,000 tokens" --> C
C -- "answer" --> B
B -- "answer" --> A
style B fill:#16a34a,stroke:#22c55e,color:#fff
Nothing changes about how you work. Same tools, same API key, same answers. Just a smaller bill.
These are real measurements, not estimates. You can run them yourself with one command (see For the nerds).
xychart-beta
title "Tokens removed, by kind of content"
x-axis ["Build logs", "JSON output", "Search results", "Stack traces", "Prose", "Code"]
y-axis "% removed" 0 --> 100
bar [88, 79, 69, 61, 44, 40]
Across a realistic mix: 79% fewer tokens.
For JSON and logs — which is where AI coding tools spend most of their budget — nothing is lost. Sensei restructures the text rather than summarising it, so the model sees the same facts.
What that means in money. If you spend €100/month on API calls, a 79% reduction on the input side is a large part of that bill. Sensei shows you the running total:
sensei stats.
1. Install
On Windows, take the installer
(sensei-setup-*.exe). It installs per-user — no administrator prompt, because
Sensei listens on loopback and only touches files in your own home directory —
adds a Start-menu entry, and offers to connect the AI tools you already have and
to start with Windows. Uninstalling puts those tools back.
Everywhere else:
pipx install sensei-gatewayOther ways in
brew install senseiissei/tap/sensei # macOS and Linux
winget install SenseiIssei.Sensei # Windows
scoop bucket add senseiissei https://github.com/SenseiIssei/scoop-bucket
scoop install sensei # Windows, portable
pip install sensei-gateway # any platform with Python 3.11+Or download a single file from the latest
release — .exe for
Windows, .dmg for macOS, .AppImage/.deb/.rpm for Linux — which needs no
Python at all.
The distribution is called sensei-gateway because sensei on PyPI is an
unrelated HTTP library from 2023. The command you type is sensei.
2. Start it
sensei up # in a terminal
sensei tray # in the background, with a system-tray icontray serves exactly what up serves, without a window you have to leave open.
Right-click the icon to open the dashboard, connect your tools, or quit. Needs
pip install "sensei-gateway[tray]"; the Windows installer includes it.
Your browser opens and asks you one question: do you want to run a model on your own computer (free), or use a service like OpenAI (paste a key). Pick one, click save. Done.
3. Point your tools at it
sensei setup-toolsSensei finds the AI tools already on your machine — Claude Code, Claude Desktop,
Cursor, Windsurf, Cline, Roo, Kilo, Zed, Continue, Codex, Aider — and configures
them. You are shown exactly which files it will touch before it touches them,
every one is backed up, and sensei setup-tools --undo puts them all back.
Only some of your work should route through it? sensei setup-tools --project
writes the configuration into the current repository instead of the machine, so
you can commit it for everyone or keep it to one checkout.
To confirm it actually worked rather than assuming:
sensei doctor --verifyThat sends a real request through the running gateway and reports what came back — separately from whether your provider is set up, because "compression is broken" and "you haven't added a key yet" are different problems with different fixes.
For a tool you launch from a terminal, there is also the no-config version, which sets one environment variable for that process and nothing else:
sensei wrap claudeSame for codex, aider, cursor-agent, cline, continue, opencode,
goose and crush — just swap the name.
4. Watch the number go up
Open http://localhost:7000/app/ and click Tokens Saved: what you saved
today, over the last thirty days, and which tool saved it. sensei stats prints
the same thing in the terminal.
The history is a SQLite file on your own disk holding counters and model names —
no prompts, no responses, nothing transmitted. SENSEI_SAVINGS_PERSIST=false
turns it off and the totals go back to being per-process.
Or use it as an MCP server
If your tool speaks MCP, Sensei plugs in directly — no base URL to change:
The agent gets three tools: compress a blob before putting it in context, get
the exact original back if it turns out it needed it, and check what's been
saved. Needs pip install "sensei-gateway[mcp]".
Prefer Docker?
git clone https://github.com/SenseiIssei/Sensei.git && cd Sensei
docker compose up -dThen open http://localhost:7000/app/.
Tokens Saved, after routing five different coding agents through the gateway. Real traffic against a local stub provider — the compression numbers are measured, the money is an estimate at your configured input price, and the history is one day because the machine was set up that morning.
The chat window, and the phone layout
Here is what the command line shows, which is real output:
$ sensei models
Your machine
System Windows 11 (AMD64)
CPU 16 cores
RAM 15.3 GB
GPU none detected — models will run on the CPU
Budget 7.7 GB usable for a model
Suggestions for this machine
Qwen2.5 Coder 7B 7B 4.6 GB fits comfortably
Code completion and review. Pairs well with the gateway.
Llama 3.2 3B 3B 2.0 GB fits comfortably
General chat on modest hardware. A sensible floor.
GLM-5.2 744B MoE 390.6 GB too large
Start here: sensei models --pull qwen2.5-coder:7bSomething not working? Sensei tells you what to do about it:
$ sensei doctor
[+] Python 3.12.10 on Windows AMD64
[+] Config file D:\Sensei\.env
[!] Ollama not installed
-> Optional. For free local inference: https://ollama.com
[x] Model access no local model and no API key — Sensei cannot answer anything
-> Either install Ollama (free, local) or run 'sensei up' and paste an
API key into the setup wizard.
[+] Port 127.0.0.1:7000 is free
[+] Bind address 127.0.0.1This matters more than the token savings, so it's worth being specific.
| 🔒 Runs only on your machine | Sensei listens on 127.0.0.1. Nothing on your network can reach it unless you deliberately change that. |
| 🚫 No telemetry, ever | No analytics, no crash reporting, no "anonymous usage data". Not in any build, not behind any flag. |
| 📡 No third-party requests | The interface loads no fonts, scripts or trackers from anyone. This is checked automatically on every commit. |
| 💾 The savings history is a local file | SENSEI_SAVINGS_PERSIST writes counters and a model name per request to a SQLite file next to Sensei. No prompt text, no responses, and nothing leaves the machine. Delete the file, or turn it off. |
| 🔑 Your key stays yours | Sensei forwards your API key to the provider you chose and to nobody else. It's encrypted on disk, never written in plain text. |
| ✂️ Secrets stripped | Optionally, Sensei removes passwords, tokens and keys from prompts before they leave your computer. |
| 📴 Works offline | If the server isn't running, the interface still opens and tells you how to start it. |
Does this make the AI worse?
For logs and JSON — the bulk of what coding tools send — no. Sensei rewrites that text into a denser form containing the same facts, rather than summarising it. For prose it removes filler ("it is important to note that" → nothing), which is the kind of text a model ignores anyway.
If the model ever does need the original, it can ask for it: Sensei keeps a local copy of everything it compressed and hands it back on request.
Do I need to change my code or my tools?
No. sensei wrap claude sets one environment variable for that program and
launches it. Nothing is written to your shell profile, nothing is installed into
your tool, and closing it puts everything back.
Do I still need an API key?
Only if you want to use a hosted model. Sensei can run entirely on your own
computer with Ollama — sensei models tells you which
models actually fit in your memory.
Is it really free?
Yes. MIT licensed, no paid tier, no feature held back. See the principles.
What if I don't like it?
pip uninstall sensei. Sensei keeps its data in dot-directories next to where
you ran it; delete them and nothing remains.
Architecture
Sensei is a FastAPI service that speaks both the OpenAI and Anthropic wire formats, so anything with a configurable base URL routes through it unchanged.
flowchart TB
subgraph client["Clients"]
CC["Claude Code<br/><sub>ANTHROPIC_BASE_URL</sub>"]
CX["Codex / Cursor / Aider<br/><sub>OPENAI_BASE_URL</sub>"]
WEB["Web UI"]
CLI["CLI · Qt app · VS Code ext"]
end
subgraph sensei["Sensei"]
GW["Gateway<br/><sub>/v1/chat/completions · /v1/messages</sub>"]
RT["ContentRouter<br/><sub>detects type, picks a compressor</sub>"]
subgraph comp["Compressors"]
SC["SmartCrusher<br/><sub>JSON → CSV schema</sub>"]
LC["LogCompressor<br/><sub>triage + dedupe</sub>"]
CD["CodeCompressor<br/><sub>comments, imports</sub>"]
TC["TextCompressor<br/><sub>filler, boilerplate</sub>"]
end
CA["CacheAligner<br/><sub>keeps the prefix byte-exact</sub>"]
CCR["CCR store<br/><sub>originals, retrievable</sub>"]
RED["DLP redaction<br/><sub>strips secrets</sub>"]
end
PROV["Provider<br/><sub>Anthropic · OpenAI · Ollama · 11 more</sub>"]
CC --> GW
CX --> GW
WEB --> GW
CLI --> GW
GW --> RT
RT --> SC & LC & CD & TC
SC & LC & CD & TC --> CA
CA --> RED
RED --> PROV
RT -.stores originals.-> CCR
CCR -.on request.-> PROV
style sensei fill:#0a0a0f,stroke:#22c55e
style comp fill:#111827,stroke:#374151
The system prompt is left byte-identical on purpose. Providers cache on an exact prefix match, so touching it would invalidate the cache and cost more in latency than compression saves.
How each compressor works
SmartCrusher — JSON. An array of objects with the same keys repeats every key name on every element. It's rewritten as a header plus rows, which is what a CSV is, and drops redundant link/self/href keys. Lossless.
[{"id":1,"name":"a","url":"…"},{"id":2,"name":"b","url":"…"}, …20 more]
↓
id|name|url
1|a|…
2|b|…
LogCompressor — build and test output. Keeps the head, the tail, every line matching an error/warning/summary pattern, and a few lines of context after each. Identical lines are collapsed after normalising timestamps, hex addresses and UUIDs, so a thousand near-identical worker lines become one with a count.
CodeCompressor — source. Strips comments, docstrings, blank lines and trailing whitespace, and folds consecutive import blocks. Regex-driven per language rather than a full parse, for speed and zero dependencies.
TextCompressor — prose. Phrase substitution ("in order to" → "to"), filler removal, boilerplate sentence stripping, and line-level dedupe.
CCR — reversible compression. Every original is written to a local cache keyed by id. If the model decides it needs the untouched text, it asks for it by id and gets it back. Compression is therefore never destructive in practice.
MCP server
sensei mcp speaks the Model Context Protocol over stdio, which covers the
tools that don't let you redirect a base URL — and the case where an agent wants
to compress one specific blob rather than route its whole conversation.
| Tool | What it does |
|---|---|
sensei_compress |
Compress text. Returns the compressed form, a ccr_id, the token counts and the percentage saved. Optionally force a compressor with content_type. |
sensei_retrieve |
Exchange a ccr_id for the byte-identical original. |
sensei_stats |
Totals saved, plus CCR cache state. |
sensei_retrieve is what makes this safe rather than lossy-by-default.
Compression is never a one-way door: if the model decides the compressed form is
missing something, it asks for the original instead of guessing. The server's
instructions tell the client exactly that, because a capability a model doesn't
know about might as well not exist.
Verified end to end against a real MCP client — 582 → 287 tokens on a 40-record JSON array, original recovered byte-identical.
pip install "sensei-gateway[mcp]"
sensei mcp # stdio; normally your client spawns this, not youPerformance
Compression sits on the hot path of every request, so its own cost matters. Measured on an 86 kB agent turn — system prompt, exchanges, a large JSON tool result, build logs and source:
| before optimisation | after | |
|---|---|---|
| median | 8.14 ms | 4.68 ms |
| p95 | 12.53 ms | 6.31 ms |
| cold start | 9.34 ms | 6.91 ms |
What that took: compiling the detector patterns once instead of per line,
bounding the type-detection scan to the sample it actually reads instead of
splitting the whole payload, and removing ~11,000 re cache lookups per request.
An optional Rust accelerator (sensei_core, PyO3, abi3) makes the JSON hot path
roughly twice as fast again. CI builds the wheel and re-runs the entire test
suite against it on every commit, so "byte-identical to the Python path" is
verified rather than asserted.
Reproducing the numbers
python backend/benchmarks/compression_benchmark.pyReal tiktoken (cl100k_base) counts over a fixed corpus of tool outputs, logs,
stack traces, source and prose. --json gives machine-readable output and
--min-aggregate 75 exits non-zero below a floor — which is what nightly CI
runs, so a regression in compression quality fails the build rather than being
noticed months later.
Every run is published: the nightly trend keeps one row per night, so 79% is a line you can look at rather than a number in a README.
Characters you pay for and cannot see
A zero-width space is one token. So is a byte-order mark, a word joiner, a soft hyphen. They arrive by the hundred when code is pasted out of a rendered web page, a wiki or a chat window, and no line-oriented compressor notices them.
$ python backend/benchmarks/invisible_benchmark.py
sample pasted clean waste removed seen
-------------------------------------------------------------------------------
code pasted from a rendered page 550 400 38% 75 0
code with a stray BOM per block 425 400 6% 25 0
markdown out of a wiki 336 336 0% 0 60
source carrying a Trojan Source override 600 500 20% 50 0
-------------------------------------------------------------------------------
AGGREGATE 1911 1636 17%Sensei removes them before counting, so the saving is already in the headline number. Nightly enforces a floor, because a stripper that stops stripping shows up as a quietly larger bill rather than as a failure.
It also removes bidi overrides and isolates — the
Trojan Source vector (CVE-2021-42574), where
source renders in one order and compiles in another, so a reviewer reads
if (isAdmin) where the compiler reads the opposite. A gateway every prompt
already passes through is the right place to catch that.
Restraint is the part worth reading. The three-line version of this feature corrupts real text:
| Zero-width joiners | Structural in Devanagari, Persian and Arabic, and what holds 👨👩👧 together as one family. Removed only from content detected as code, where they cannot mean anything. |
| Directional marks (LRM, RLM) | Ordinary punctuation in Hebrew and Arabic prose. Kept. Only the overrides go. |
| Non-breaking spaces | Deliberate in typeset prose. Counted, never rewritten, unless you set SENSEI_STRIP_NBSP=true — and then to a space, not to nothing. |
| Homoglyphs | раssword with a Cyrillic а is a different identifier than it looks. Logged, never silently corrected: "almost always an attack" is not a licence to edit your text. |
The untouched original stays in the CCR store, byte-identical, retrievable by
id — SENSEI_STRIP_INVISIBLE=false turns the whole thing off.
Shorter answers, and how we know
Compression only touches what goes up. Output tokens cost roughly 4-5× input tokens, so the other half of the bill was untouched until now.
SENSEI_OUTPUT_SHAPER=true sensei upThis appends a short instruction asking for answers without preamble, restated questions or closing offers of further help. It goes on the last user message, never the system prompt — the system prompt is the cached prefix, and appending to it would invalidate the provider's cache on every request and cost more in latency than the shaping saves.
It ships with a holdout, and that is the point. An intervention that changes
model behaviour cannot be measured by switching it on and reading the number:
the number moves for a dozen other reasons. So SENSEI_OUTPUT_HOLDOUT (default
10%) leaves a fraction of requests unshaped, and sensei stats compares the two
groups:
Output shaping
Shaped 360 requests, 281.4 tokens/answer on average
Control 40 requests, 402.7 tokens/answer on average
95% confident the change is between 22.5% and 37.5%
Best estimate: 30.1% (121.3 tokens/answer)
Verdict: shorter answersUntil both groups have enough samples it says "not enough data yet" and reports no percentage at all. A confident-looking figure computed from eleven requests is worse than no figure, because that is the one that gets quoted.
Off by default: it changes what the model writes, which is your call. Streaming replies are excluded — they carry no usage block, and reconstructing one would mean parsing the stream on the hot path.
Does it keep what the model needs?
A different question from "does it get smaller", and until recently only the second one was checked anywhere here.
python backend/benchmarks/quality_eval.pyEach corpus entry is paired with the facts an agent would have to extract from it — the error location in a build log, the failing frame in a stack trace, specific ids and values in JSON, a function signature in source. All of them must still be literally present after compression. Nightly runs it with a floor of 100%, because unlike a savings percentage a lost fact is never jitter.
Be clear about the claim: this is a necessary condition, not a sufficient
one. If a fact is gone the model provably cannot answer; if every fact
survives the model can answer, but this does not prove it will. The
sufficient version needs a model in the loop — --model gpt-4o --base-url ...
does that, and it is deliberately not part of the nightly gate, because a gate
that needs a funded API key is a gate somebody eventually switches off.
Writing this eval immediately found two real defects: LogCompressor.FRAME
contained File ", " where File " was meant, so no Python traceback frame
ever matched and the innermost frames — the ones that say what actually broke —
were being elided. And build output with no timestamps and no log levels was
being classified as prose, which both compressed it 3% instead of 91% and threw
away the compiler's file:line:col.
Security model
Sensei assumes the machine it runs on is trusted and that anyone who can reach the port is authorised. Everything else follows from that:
- Binds to
127.0.0.1. Exposing it is a deliberate act, andsensei doctorreports it as a failure if you do it without enabling auth. - API keys live in an encrypted vault (AES-256-GCM), not in
.env. - Optional DLP redaction strips API keys, tokens, private keys, JWTs and optionally PII from prompts before they leave the machine.
- Optional per-user auth, JWT sessions, OIDC SSO, and RBAC on admin endpoints.
run_pythonis off by default. When enabled it is a subprocess with a timeout, not a container — the docs say so plainly.
Full scope and reporting process: SECURITY.md.
Repository layout
backend/ FastAPI service, compression pipeline, CLI, tests
sensei/
compression/ SmartCrusher, LogCompressor, CodeCompressor, TextCompressor,
CacheAligner, CCR, ContentRouter
routers/ gateway, chat, RAG, agent, settings, setup, stats, …
security/ auth, crypto, vault, redaction, RBAC, OIDC, sessions
integrations writes Sensei into other tools' config files, reversibly
savings.py in-memory totals + the local SQLite savings ledger
cli/ up · wrap · setup-tools · doctor · models · stats · chat
benchmarks/ the compression benchmark
frontend/ React 19 + Tailwind 4 + Vite 8, PWA
rust/sensei_core/ optional PyO3 accelerator
extensions/vscode VS Code extension
training/ Sensei-Compressor fine-tuning scaffold
deploy/ nginx · Caddy · Traefik · systemd
docs/ configuration reference (generated), the plan
What CI actually enforces
Every action is pinned to a commit SHA. On each pull request:
- backend tests across {Linux, macOS, Windows} × Python {3.11, 3.12, 3.13}
rufflint and format, with the rule set pinned so a new ruff release can't fail an unrelated PR- the Rust accelerator built and the whole suite re-run against it to prove byte-parity
- Playwright on three operating systems
- a 450 kB JavaScript budget
- the config reference regenerated from the pydantic model and compared
- the built UI checked for third-party origins, and for root-absolute asset
URLs that would blank the page when mounted under
/app/ - Docker image built and
/healthsmoke-tested - pip-audit, npm audit, cargo audit, CodeQL, gitleaks, Trivy, SBOM, Scorecard
Nightly: the compression benchmark with a regression floor, fact retention at a floor of 100%, Rust/Python output diffed for parity, the suite against eagerly-upgraded dependencies, and the results appended to the public trend.
Sensei didn't invent any of this. Two projects did the hard thinking, and both are worth your time independently of whether you ever use Sensei:
🪶 Headroom · Apache-2.0
The reason this project exists. Headroom worked out that the way to cut an agent's token bill isn't to summarise its context — it's to notice that most of what agents send is structurally redundant, and restructure it losslessly. Tabular JSON compaction, log triage, cache-aligned prefixes, and reversible compression with a retrieval tool are all Headroom's ideas.
If you want context compression as a mature, standalone product with a much larger surface than Sensei's — output-token shaping, a learned prose compressor, LangChain and LiteLLM adapters, agent wrapping for a dozen tools — go use Headroom. It's excellent, and it's further along than this.
Sensei's angle is different: compression as one part of a self-hosted workspace you own end to end, MIT-licensed, that also gives you a chat UI, RAG, an agent and a desktop app.
🏛️ Odysseus · AGPL-3.0
The shape of the thing. Odysseus — by Felix Kjellberg — made the case that a
self-hosted AI workspace should feel like a finished product rather than a pile
of scripts: hardware-aware model recommendations, a real editor, agents, memory,
and security defaults that assume you'll actually deploy it. The "what can my
machine actually run?" idea behind sensei models is straight from its Cookbook.
Sensei is a clean-room implementation — no Odysseus code is copied, which is what lets it stay MIT instead of inheriting AGPL. If you want the full workspace with email, calendar, documents and image editing, Odysseus is the more complete answer.
Neither project endorses this one. Any bugs here are ours.
Also worth knowing: GLM-5.2 is the model Sensei's defaults point at, and the reason a lot of the compression tuning assumes a very large context window.
Issues and PRs welcome — CONTRIBUTING.md has the setup.
docs/ROADMAP-NEXT.md is the queue — what's planned, in what order, and why. It also lists the things that cost real time to discover and aren't visible from reading the code.
Most useful right now: compression heuristics for more languages, output-token shaping, mobile polish, and screenshots for this page.
{ "mcpServers": { "sensei": { "command": "sensei", "args": ["mcp"] } } }