Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
24 changes: 23 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Changelog

## 1.0.1-beta.0 — 2026-08-12
## 1.0.1-beta.0 — 2026-08-14

### Features

Expand All @@ -22,6 +22,28 @@

- Make the canary box a one-command install. Setting it up was four commands, and three of them fail SILENTLY for a day — the wrong property for the thing whose whole job is noticing silent failures. A work dir mounted at a different path inside the container than out leaves the sibling-container `-v` sources resolving against the host to nothing; a `CANARY_REF` left at the shipped `origin/failproofaid` points the box at a branch that merged in #632, so it would test a frozen tree forever and never say so; and a filled-in env file with no Slack webhook produces a run that works perfectly and reports nowhere, which is worse than no canary because it looks like coverage. `integration-suite/local/install.sh` refuses each at install time, in front of a person, rather than at 06:17 tomorrow in front of nobody — the webhook is required for that reason, not because the run needs it. It builds the runner image straight from the git URL (Docker takes `<repo>#<ref>:<subdir>` as a build context) so the box never clones, installs the env file at mode 600, and REWRITES rather than appends its cron line — it carries a `# failproofai-canary` marker and strips any previous line first, so re-running upgrades the schedule instead of scheduling a second job. No credentials template ships in the repo at all — a file that looks like a credentials file is one `git add -A` away from being committed by whoever fills it in — so running the installer with no arguments prints the variable list instead, generated from the same `REQUIRED_` lists it enforces and therefore unable to drift the way a checked-in example silently does. `--dry-run` distinguishes what it CHECKED (the preflight really runs; it keeps its ✓) from what it would CHANGE, because a script reporting success for work it did not do is the same defect class this canary exists to find. (#686)
- Stop the nightly doc translation re-translating everything, most days. Runs cost **4 minutes** on Aug 3-5 and **118-136 minutes** every day from Aug 6-11 — ~750 wasted runner-minutes and six full-corpus passes through the LLM gateway in six days. Three causes compound, and none of them was the translation cache's own logic, which is sound. **First, the cache was being evicted between runs.** `ci.yml` cached `target/` under a combined `actions/cache@v6`, so every PR ref that missed the exact key wrote its own 1.5-2.3 GiB copy; five were live at once (#677, #679, #680, #681 and main), putting the repo at **11.56 GiB against GitHub's 10 GiB cap** and so permanently in LRU eviction. What that evicted was the 13 KB translation cache — touched once every 24 hours, therefore always the least-recently-used thing in the store. The restore/save split is the one `build-daemon.yml:117-144` already uses, and its comment there already gives the second reason to want it. **Second, the cache was saved once, at the end of a serial pipeline.** The only save sat in `consolidate`, downstream of both the matrix gate and `mintlify validate`, so a single page failing in a single language discarded all fourteen languages' work: Aug 6 lost ~110 completed minutes to one `ko` page. Each language now saves its own fragment in the job that produced it, immediately after the step that proved it good; the merged entry stays as a cross-language fallback. **Third, a cache HIT never checked that the translated file exists.** `isCached` is a pure function of the English source hash — it records that a page was translated once, not that it is on disk — and translations land on an auto-translate PR branch. With #682 unmerged, `main` lacked `docs/<lang>/cli/{update,migrate}.mdx` while the cache reported them done, so they were never regenerated, `--update-nav` (which reads the *English* tree) emitted nav entries pointing at them, and `mintlify validate` failed on 28 missing files. That is non-convergent: **a cache hit fails validation and only a full 120-minute miss goes green**, which is exactly what Aug 12 did. Statting the output makes the cache self-healing against any "translated once, never landed" gap. Also: a cache miss is now a visible `::warning` rather than silent — the old restore key always evaluated to the bare literal `translation-cache-`, since the file is gitignored and `hashFiles` returns `""` for an absent path, so every restore that ever worked was a prefix fallback and a total miss looked identical to a hit. Artifact retention goes 1 → 7 days so a run that dies mid-pipeline leaves a manual recovery path. (#685)
### Docs

- Reduced Cloud onboarding to four tasks: send a trace (including backfill), run an audit, author a policy, and deploy it. The landing page and local quickstart now use the same labels, policy authoring and deployment have separate pages, and the primary connection section lists supported agents followed by the Python SDK instead of exposing collector implementation pages. (#687)
- Tightened the onboarding and Cloud documentation: shorter paragraphs, direct feature descriptions, consistent Issues terminology, and a clearer split between the local quickstart and Cloud setup. Fixed localized onboarding links that used an English-only heading anchor. (#687)
- Reordered the documentation around the reliability loop, cloud-first. The landing page opened "FailproofAI is the policy layer" and treated the cloud as a second half; it is now [What is FailproofAI](https://docs.befailproof.ai/introduction) — observe every run, score it, find the failures nobody wrote a rule for, prevent them, verify the next audit is clean. [How it works](https://docs.befailproof.ai/how-it-works) and [Concepts](https://docs.befailproof.ai/concepts) follow the same order. The sidebar splits into a **Documentation** tab (the journey) and a **Reference** tab (14 CLI pages, files, env vars), which takes 28 pages out of the first sidebar a new reader scrolls. (#687)
- Added the cloud onboarding path, which did not exist. Getting an API key was an off-site link on four pages, and "send your first trace" lived only inside the SDK reference. Four sequential pages now cover [sending a trace](https://docs.befailproof.ai/start/first-trace), [running an audit](https://docs.befailproof.ai/start/first-audit), [authoring a policy](https://docs.befailproof.ai/start/first-policy), and [deploying it](https://docs.befailproof.ai/start/deploy-policy). Workspace creation is linked as a prerequisite, and backfill is part of the trace step. (#687)
- Made the two ways to plug in explicit everywhere instead of a buried note: agents on a **supported harness** connect with `failproofai config --connect` and need no code change; **custom agents you wrote** stream events through the Python SDK. (#687)
- Renamed four cloud pages to match what the dashboard actually calls them, with redirects in all 15 language trees: `/cloud/incidents` → [`/cloud/issues`](https://docs.befailproof.ai/cloud/issues), `/cloud/managed-policies` → [`/cloud/policy-editor`](https://docs.befailproof.ai/cloud/policy-editor), `/cloud/fleet` → [`/cloud/deploy-policies`](https://docs.befailproof.ai/cloud/deploy-policies), and `/cloud/access` → [`/cloud/admin`](https://docs.befailproof.ai/cloud/admin). 35 existing redirects pointed at two of those old destinations and were repointed. (#687)
- Rewrote the cloud overview as [What's in the dashboard](https://docs.befailproof.ai/cloud/overview): one table mapping every sidebar item to its page. It previously enumerated an Observe/Analyze/Govern sidebar the product no longer has, and four other pages named that retired grouping in prose. (#687)
- Documented the two admin surfaces that had no docs — **Usage** (the fixed 30-day metering window, `usage:read`) and **Settings** (default alert recipients, model context windows) — and the two cloud CLI command groups that were missing from a page claiming to list all of them: `agenteye audits` and `agenteye usage`. The CLI's issue group is `issues`, not `incidents`; the page said `incidents`. (#687)
- Killed the two name collisions the glossary had been apologising for: the local one-shot scan is now **Local audit** against the fleet-wide **Audits**, and the local UI is **Local dashboard** against **Charts and boards**. Evaluations became **Online evals** and Evaluators **Build an evaluator**, so the benefits page and the build guide no longer share a name. (#687)
- Stated the `agenteye` package name where a reader meets it, on the SDK and cloud CLI pages, rather than leaving two products' worth of identifiers unexplained. `failproofai config --help` said "FAILPROOF CLOUD", a third spelling; it now says FailproofAI Cloud. (#687)
- Rebuilt the documentation as one product instead of two. The site had two top-level tabs — "Enforcement" and "Observability" — which asked every reader to work out, before reading anything, which half of a single product they were in. There is now one navigation, ordered as a journey: install and guard a machine, see what happened on it, then govern the fleet from the cloud. The observability section is repositioned as **FailproofAI Cloud** and lives at `/cloud/*` (was `/agenteye/*`), with redirects from every old URL. (#687)
- Documented the integration surface that had no docs at all. `failproofai config --connect` was described only in `--help`: new pages cover [connecting a machine](https://docs.befailproof.ai/cloud/connect) (both capabilities, what leaves the machine, fleet provisioning, troubleshooting), [managed policies](https://docs.befailproof.ai/cloud/managed-policies) (deployments, digest verification, observe-mode rollout), [the fleet view](https://docs.befailproof.ai/cloud/fleet), and [session capture](https://docs.befailproof.ai/cloud/capture) across all 12 CLIs — replacing three per-CLI capture pages written for a separate collector. (#687)
- Added the pages a reader kept needing and could not find: [How it works](https://docs.befailproof.ai/how-it-works) (tool call → decision → dashboard, end to end, with the failure-mode table), [the failproofaid service](https://docs.befailproof.ai/daemon) (fail-closed, supervision, how the binary arrives, upgrades), [Supported agents](https://docs.befailproof.ai/agent-support) (per-CLI matrix of what a deny can actually block — the one fact that decides whether a policy does anything), [Policies](https://docs.befailproof.ai/policies) as a hub, one merged [Concepts](https://docs.befailproof.ai/concepts) glossary, and a [files and paths](https://docs.befailproof.ai/reference/files) reference. New CLI pages for `config`, `harness`, `backfill`, `flush`, and `uninstall`. (#687)
- Moved per-CLI hook-schema internals out of the Configuration page and into the support matrix, so configuration is about configuring again rather than a wall of vendor contracts. (#687)
- Fixed the migration this rebuild left behind. Every removed URL now redirects: the 14 localized trees had none of the 57 redirects (every non-English bookmark 404'd), and 13 redirects that were live on `main` had been rewritten at the *source* rather than the destination, so the URLs people actually hold stopped resolving. Redirect coverage is now complete in all 15 trees, and 1,792 links in translated pages that pointed at English paths were localized. (#687)
- Corrected claims that did not match the code. The built-in policy count (40, not 39) and the Recommended preset (15, not 16); the hook exit codes (deny is 2, instruct is 0); `credentials.json`, `audit/cache/` and `logs/hooks.log` paths; the deny message users actually see; that the activity log records allows too; and that `state/` holds the undelivered upload spool and must not be deleted. (#687)
- Stopped overstating what the sanitize policies do. They detect secrets in tool output and, on ten of the twelve CLIs, cannot remove them — the `PreToolUse` policies `block-env-files`, `protect-env-vars` and `block-secrets-write` are what actually keep a secret out. The turn-end gap on Hermes and Goose is likewise a scope decision, not a platform limit. (#687)
- Made the files reference match a real machine. Each path now says when it appears, because a fresh install creates seven entries and everything else — `migrations/`, `credentials.json`, `audit/`, `logs/` — is conditional. (#687)
- Completed the agent-skills page: it documented three skills where four ship, omitting `failproofai-policy-author`. (#687)
- Replaced "guardrails" with "policies" throughout, including the navigation group and the transliterated forms in every translated tree. (#687)

## 1.0.0 — 2026-08-12

Expand Down
8 changes: 5 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -188,12 +188,14 @@ when something goes wrong. → [Dashboard guide](https://docs.befailproof.ai/das

| | |
|---|---|
| [Getting Started](https://docs.befailproof.ai/getting-started) | Installation and first steps |
| [Built-in Policies](https://docs.befailproof.ai/built-in-policies) | All 30 policies with parameters |
| [Quickstart](https://docs.befailproof.ai/quickstart) | Installation and first steps |
| [How it works](https://docs.befailproof.ai/how-it-works) | Tool call → decision → dashboard, end to end |
| [Built-in Policies](https://docs.befailproof.ai/built-in-policies) | All 39 policies with parameters |
| [Custom Policies](https://docs.befailproof.ai/custom-policies) | Write your own |
| [Configuration](https://docs.befailproof.ai/configuration) | Config scopes and merge rules |
| [Supported agents](https://docs.befailproof.ai/agent-support) | All 12 agent CLIs, and what each can block |
| [Dashboard](https://docs.befailproof.ai/dashboard) | Session monitor and policy activity |
| [Architecture](https://docs.befailproof.ai/architecture) | How the hook system works |
| [FailproofAI Cloud](https://docs.befailproof.ai/cloud/overview) | Fleet-wide policy, observability, and evaluation |

---

Expand Down
4 changes: 2 additions & 2 deletions __tests__/scripts/translate-docs/mdx-translator.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -60,10 +60,10 @@ function emptyCache(): TranslationCache {
}

describe("getEnglishMdxPages", () => {
it("includes AgentEye pages in automatic translation", () => {
it("includes cloud pages in automatic translation", () => {
const pages = getEnglishMdxPages();
expect(pages.length).toBeGreaterThan(0);
expect(pages.some((page) => page.includes("/agenteye/"))).toBe(true);
expect(pages.some((page) => page.includes("/cloud/"))).toBe(true);
});
});

Expand Down
10 changes: 5 additions & 5 deletions __tests__/scripts/validate-mdx.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -253,7 +253,7 @@ describe("findBrokenAssetRefs", () => {
// Fixtures resolve against the real repo so the check is exercised with the
// same two path conventions the docs actually use.
const REPO = join(__dirname, "..", "..");
const DOCS_PAGE = join(REPO, "docs", "agenteye", "alerts.mdx");
const DOCS_PAGE = join(REPO, "docs", "cloud", "alerts.mdx");
const I18N_PAGE = join(REPO, "docs", "i18n", "README.ja.md");

it("flags the exact regression that broke every translated README", () => {
Expand Down Expand Up @@ -281,19 +281,19 @@ describe("findBrokenAssetRefs", () => {
});

it("resolves a leading slash against docs/, not the page directory", () => {
// Mintlify site-absolute form, used by every agenteye page.
// Mintlify site-absolute form, used by every cloud page.
expect(
findBrokenAssetRefs(
DOCS_PAGE,
"![Alerts](/agenteye/images/alerts.png)\n",
"![Alerts](/cloud/images/alerts.png)\n",
),
).toEqual([]);
const broken = findBrokenAssetRefs(
DOCS_PAGE,
"![Nope](/agenteye/images/does-not-exist.png)\n",
"![Nope](/cloud/images/does-not-exist.png)\n",
);
expect(broken).toHaveLength(1);
expect(broken[0].resolved).toBe("docs/agenteye/images/does-not-exist.png");
expect(broken[0].resolved).toBe("docs/cloud/images/does-not-exist.png");
});

it("checks srcset candidates, not just src", () => {
Expand Down
2 changes: 1 addition & 1 deletion bin/failproofai.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -1564,7 +1564,7 @@ WHAT IT DOES
3. Policies — presets (combine any), Everything, or a custom pick
4. Review — confirms the exact files it will change, then applies

FAILPROOF CLOUD
FAILPROOFAI CLOUD
failproofai config --connect <url> --token <key> [--machine-id <id>]
Connect this machine to FailproofAI Cloud
[--no-transcripts] decisions only, no transcripts
Expand Down
Loading