-
Notifications
You must be signed in to change notification settings - Fork 35
Move the integration suite, doc translation and a new weekly docs audit onto one cron box #694
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
35b1f03
245ac39
22010ae
a2d206e
30f9807
4cac6b4
ef2bfd6
68e8014
2011856
2c5a967
9dd17e0
f45a9c7
8a78962
7a1333b
757dfc7
8ac4a8d
1aa3cea
fb21d18
dfd8957
cf4f057
02ab2db
84652e7
4eab163
f4a99bf
bcc94a3
8a306af
53a27a9
d38a6c4
ca9e065
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,122 @@ | ||
| name: build-canary-runner | ||
|
|
||
| # Builds and pushes the canary box's RUNNER image to GHCR. | ||
| # | ||
| # The box runs three scheduled jobs — the CLI integration suite, the nightly doc | ||
| # translation, and the weekly docs audit — and this is the one image all three | ||
| # share. Publishing it is what lets an operator set the box up with nothing but | ||
| # Docker and a credentials file: no clone, no build, no installer. | ||
| # | ||
| # docker run --rm --pull=always -e CANARY_JOB=docs-audit \ | ||
| # -e CANARY_WORK="$HOME/fp-canary" -v "$HOME/fp-canary:$HOME/fp-canary" \ | ||
| # --env-file "$HOME/fp-canary.tokens" \ | ||
| # ghcr.io/failproofai/failproofai-canary-runner:latest | ||
| # | ||
| # THE IMAGE IS A TOOLCHAIN AND NOTHING ELSE — node, bun, git, the docker client | ||
| # and mintlify. It carries no credentials and no repo checkout: every job clones | ||
| # the repo itself at run time, and every secret arrives through --env-file. That | ||
| # is what makes it safe to publish publicly, which in turn is what keeps the | ||
| # operator's cron line free of a `docker login` and a fourth expiring token. | ||
| # | ||
| # Path-filtered, because the image only needs rebuilding when the baked layer | ||
| # changes. Job scripts live in the repo and reach the box through that run-time | ||
| # clone, so they must NOT trigger a publish — that split is the whole reason a | ||
| # harness change never asks anyone to touch the box. | ||
|
|
||
| on: | ||
| push: | ||
| branches: [main] | ||
| paths: | ||
| - 'integration-suite/local/Dockerfile.runner' | ||
| - 'integration-suite/local/runner-entrypoint.sh' | ||
| - '.github/workflows/build-canary-runner.yml' | ||
| workflow_dispatch: | ||
| inputs: | ||
| tag_suffix: | ||
| description: 'Extra tag alongside :latest and :sha-<short> (e.g. dev). Allowed chars: [A-Za-z0-9._-], max 128. Empty for none.' | ||
| default: '' | ||
| required: false | ||
| push_to_ghcr: | ||
| description: 'Push to GHCR. Uncheck to build-only (validate the Dockerfile without publishing).' | ||
| type: boolean | ||
| default: true | ||
| required: false | ||
|
|
||
| permissions: | ||
| contents: read | ||
| packages: write | ||
|
|
||
| concurrency: | ||
| group: build-canary-runner-${{ github.ref }} | ||
| cancel-in-progress: false | ||
|
|
||
| jobs: | ||
| build: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | ||
| with: | ||
| persist-credentials: false | ||
|
|
||
| - name: Set up Docker Buildx | ||
| uses: docker/setup-buildx-action@bb05f3f5519dd87d3ba754cc423b652a5edd6d2c # v4.2.0 | ||
|
|
||
| - name: Log in to GHCR | ||
| uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0 | ||
| with: | ||
| registry: ghcr.io | ||
| username: ${{ github.actor }} | ||
| password: ${{ secrets.GITHUB_TOKEN }} | ||
|
|
||
| - name: Compute tags | ||
| id: tags | ||
| env: | ||
| TAG_SUFFIX: ${{ inputs.tag_suffix }} | ||
| run: | | ||
| short_sha="${GITHUB_SHA::7}" | ||
| if [ -n "$TAG_SUFFIX" ]; then | ||
| if ! printf '%s' "$TAG_SUFFIX" | grep -qE '^[A-Za-z0-9_.-]{1,128}$'; then | ||
| echo "::error::tag_suffix '$TAG_SUFFIX' has invalid chars; allowed: [A-Za-z0-9._-], max 128" | ||
| exit 1 | ||
| fi | ||
| fi | ||
| { | ||
| echo "tags<<EOF" | ||
| echo "ghcr.io/failproofai/failproofai-canary-runner:latest" | ||
| echo "ghcr.io/failproofai/failproofai-canary-runner:sha-${short_sha}" | ||
| if [ -n "$TAG_SUFFIX" ]; then | ||
| echo "ghcr.io/failproofai/failproofai-canary-runner:${TAG_SUFFIX}" | ||
| fi | ||
| echo "EOF" | ||
| } >> "$GITHUB_OUTPUT" | ||
|
|
||
| - name: Build and push | ||
| uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0 | ||
| with: | ||
| context: integration-suite/local | ||
| file: integration-suite/local/Dockerfile.runner | ||
| push: ${{ github.event_name != 'workflow_dispatch' || inputs.push_to_ghcr }} | ||
| tags: ${{ steps.tags.outputs.tags }} | ||
| cache-from: type=gha | ||
| cache-to: type=gha,mode=max | ||
| provenance: false | ||
|
|
||
| # The first publish creates the package PRIVATE, and a private package | ||
| # turns the operator's one-line cron into a `docker login` plus a fourth | ||
| # credential that expires and silently breaks every job when it does. | ||
| # There is nothing in these layers to protect — see the header — so this | ||
| # flips it once and is a no-op on every run after. | ||
| - name: Make the package public | ||
| if: ${{ github.event_name != 'workflow_dispatch' || inputs.push_to_ghcr }} | ||
| continue-on-error: true | ||
| env: | ||
| GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| run: | | ||
| gh api -X PATCH \ | ||
| -H "Accept: application/vnd.github+json" \ | ||
| "/orgs/failproofai/packages/container/failproofai-canary-runner" \ | ||
| -f visibility=public \ | ||
| && echo "package is public" \ | ||
| || echo "::warning::could not set visibility — set it once by hand at | ||
| https://github.com/orgs/FailproofAI/packages, or the box needs a docker login" | ||
|
Comment on lines
+110
to
+122
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -eu
printf '%s\n' '--- workflow section ---'
sed -n '1,180p' .github/workflows/build-canary-runner.yml
printf '%s\n' '--- related visibility and package references ---'
rg -n -C 3 'visibility|failproofai-canary-runner|push_to_ghcr|workflow_dispatch|packages:|permissions:' .github README.md docs 2>/dev/null || trueRepository: FailproofAI/failproofai Length of output: 27067 🏁 Script executed: #!/bin/bash
set -u
cat > /tmp/original-visibility.sh <<'SH'
gh api -X PATCH \
-H "Accept: application/vnd.github+json" \
"/orgs/failproofai/packages/container/failproofai-canary-runner" \
-f visibility=public \
&& echo "package is public" \
|| echo "::warning::could not set visibility"
SH
cat > /tmp/proposed-visibility.sh <<'SH'
gh api -X PATCH \
-H "Accept: application/vnd.github+json" \
"/orgs/failproofai/packages/container/failproofai-canary-runner" \
-f visibility=public \
echo "package is public"
SH
mkdir -p /tmp/fake-gh-success /tmp/fake-gh-failure
cat > /tmp/fake-gh-success/gh <<'SH'
#!/bin/bash
printf 'gh args:'
printf ' <%s>' "$@"
printf '\n'
exit 0
SH
cat > /tmp/fake-gh-failure/gh <<'SH'
#!/bin/bash
printf 'gh args:'
printf ' <%s>' "$@"
printf '\n'
exit 1
SH
chmod +x /tmp/fake-gh-success/gh /tmp/fake-gh-failure/gh
for script in original proposed; do
printf '\n--- %s with gh success ---\n' "$script"
PATH="/tmp/fake-gh-success:$PATH" bash "/tmp/${script}-visibility.sh"
printf 'status=%s\n' "$?"
printf '\n--- %s with gh failure ---\n' "$script"
PATH="/tmp/fake-gh-failure:$PATH" bash "/tmp/${script}-visibility.sh"
printf 'status=%s\n' "$?"
doneRepository: FailproofAI/failproofai Length of output: 1085 🌐 Web query:
💡 Result: Currently, there is no GitHub REST API endpoint available to programmatically change a package's visibility (e.g., from private to public) in the GitHub Container Registry (GHCR) [1][2]. Because this functionality is not exposed via the API, users must manage package visibility manually through the GitHub web interface [3][1]. To change a package's visibility: 1. Navigate to the package's landing page on GitHub [3]. 2. Click on "Package settings" (typically found on the right-hand side) [3]. 3. Scroll to the "Danger Zone" at the bottom of the page and select "Change visibility" [3]. 4. Choose the desired visibility setting and confirm the change [3]. Important considerations regarding GitHub Packages visibility and access: - Irreversibility: Once a package is made public, it cannot be changed back to private [3]. - Default Visibility: New packages default to private, but they may inherit permissions from linked repositories or organization-level settings [3][4][5]. It is recommended to configure organization-level defaults to ensure new packages are private by default [5]. - GITHUB_TOKEN and Permissions: While the GITHUB_TOKEN is the recommended method for authentication in GitHub Actions, it governs access control and operations (like pushing or pulling) based on repository permissions and package settings, not the modification of the package's visibility itself [6][5][7]. The packages:write permission is required for workflows to upload new versions to a package [6][7]. - Anonymous Pulls: If a package is set to public, it is pullable by anyone on the internet, anonymously [5][7]. Organizations should regularly audit their existing public packages to avoid accidental exposure of internal details [5]. Citations:
🏁 Script executed: #!/bin/bash
set -eu
python3 - <<'PY'
import json
import urllib.request
urls = [
"https://raw.githubusercontent.com/github/rest-api-description/main/descriptions/api.github.com/api.github.com.json",
"https://docs.github.com/public/openapi.json",
]
for url in urls:
print(f"--- {url} ---")
try:
with urllib.request.urlopen(url, timeout=20) as response:
data = json.load(response)
paths = data.get("paths", {})
matches = [
path for path in paths
if "packages" in path.lower()
and ("visibility" in path.lower() or "container" in path.lower())
]
print("\n".join(matches) or "(no matching package visibility/container paths)")
except Exception as exc:
print(f"ERROR: {exc}")
PYRepository: FailproofAI/failproofai Length of output: 620 🌐 Web query:
💡 Result: There is no direct GitHub REST API endpoint to change the visibility of a package in the Container registry (or other GitHub Packages registries) [1][2]. While the GitHub REST API provides endpoints to list, delete, and restore packages, updating package settings—such as changing visibility—is not supported through the API [1][3][4]. To change a package's visibility, you must use the GitHub web interface: 1. Navigate to the package's landing page on GitHub. 2. Click on Package settings in the right-hand sidebar [5][6]. 3. Scroll down to the Danger Zone section [5][6]. 4. Click Change visibility and select the desired setting (Public, Private, or Internal) [5][6]. Important considerations: - Once a package is made public, it cannot be changed back to private [5][6]. - If a package is linked to a repository, it typically inherits the repository's access permissions by default [7][3]. You may need to remove this link or explicitly configure granular permissions to manage the package's visibility independently [7][3]. Citations:
Handle GHCR visibility outside this workflow. GitHub REST API does not expose a package-visibility update endpoint, so this 🤖 Prompt for AI Agents |
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,17 +1,24 @@ | ||
| name: Translate Docs | ||
|
|
||
| on: | ||
| # Auto-translation used to run on every push to main that touched a | ||
| # translatable source, fanning out the full 14-language matrix per doc | ||
| # commit — expensive. We batch instead: one daily run at 11:05 IST | ||
| # (05:35 UTC — GitHub Actions cron is always UTC) coalesces a day's | ||
| # English-source edits. The content-hash cache | ||
| # (scripts/translate-docs/.translation-cache.json) still limits token spend to | ||
| # the documents whose source actually changed since the last successful run, | ||
| # so most days translate only a handful of pages (or none). Use the manual | ||
| # workflow_dispatch below for an on-demand or forced re-translation. | ||
| schedule: | ||
| - cron: "35 10 * * *" # 11:05 IST (05:35 UTC) | ||
| # ON-DEMAND FALLBACK. The NIGHTLY translation moved off Actions to the local | ||
| # box for cost — runner minutes were its entire expense, and the LLM spend is | ||
| # identical wherever it runs. integration-suite/local/jobs/translate.sh is the | ||
| # job that replaced the schedule that used to live here; the box runs it at | ||
| # 02:00 local, and integration-suite/local/install.sh sets it up. | ||
| # | ||
| # This workflow stays dispatch-only: the cloud escape hatch for when the box | ||
| # is down, or when a clean cloud reproduction is wanted. Note that its | ||
| # Actions-cache state is SEPARATE from the box's cache file, so a dispatch | ||
| # may re-translate pages the box already has (costing a full pass, not a | ||
| # wrong result). | ||
| # | ||
| # History, since it explains the shape below: auto-translation once ran on | ||
| # every push to main that touched a translatable source, fanning the full | ||
| # 14-language matrix out per doc commit. Batching to one daily run coalesced | ||
| # a day's English-source edits, and the content-hash cache | ||
| # (scripts/translate-docs/.translation-cache.json) limits token spend to the | ||
| # documents whose source actually changed. | ||
| workflow_dispatch: | ||
| inputs: | ||
| force: | ||
|
|
@@ -80,12 +87,32 @@ jobs: | |
| # hook, which builds the full Next.js application once per language. | ||
| run: bun install --frozen-lockfile --ignore-scripts | ||
|
|
||
| # The old primary key was | ||
| # `translation-cache-${{ hashFiles('scripts/translate-docs/.translation-cache.json') }}`, | ||
| # which ALWAYS evaluated to the bare literal `translation-cache-`: the file | ||
| # is gitignored (.gitignore:68), so it is absent at checkout and | ||
| # `hashFiles` returns "". Every restore that ever worked was a | ||
| # `restore-keys` prefix match, and a total miss is indistinguishable from a | ||
| # hit — nothing fails, nothing warns, the job just spends nine minutes and | ||
| # a full LLM pass. Hence the explicit warning step below: a miss is the | ||
| # expensive case and it should say so in the run summary. | ||
| - name: Restore translation cache | ||
| id: restore-cache | ||
| uses: actions/cache/restore@v6 | ||
| with: | ||
| path: scripts/translate-docs/.translation-cache.json | ||
| key: translation-cache-${{ hashFiles('scripts/translate-docs/.translation-cache.json') }} | ||
| restore-keys: translation-cache- | ||
| # Per language, newest-first, falling back to the merged entry that | ||
| # `consolidate` still writes. `github.run_id` is monotonic, so the | ||
| # prefix match returns this language's most recent fragment. | ||
| key: translation-cache-${{ matrix.lang }}-${{ github.run_id }} | ||
| restore-keys: | | ||
| translation-cache-${{ matrix.lang }}- | ||
| translation-cache- | ||
|
|
||
| - name: Warn on translation cache miss | ||
| if: steps.restore-cache.outputs.cache-matched-key == '' | ||
| run: | | ||
| echo "::warning title=Translation cache MISS::${{ matrix.lang }} will re-translate every page (~9 runner-minutes and one full LLM pass)" | ||
|
Comment on lines
+112
to
+115
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔒 Security & Privacy | 🟠 Major | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/usr/bin/env bash
set -euo pipefail
rg -n -C 5 'prepare:|languages|workflow_dispatch|matrix\.lang|cache-matched-key' \
.github/workflows/translate-docs.yml .github/workflowsRepository: FailproofAI/failproofai Length of output: 34673 🏁 Script executed: #!/usr/bin/env bash
set -euo pipefail
printf '%s\n' '--- workflow header and preparation ---'
sed -n '1,75p' .github/workflows/translate-docs.yml
printf '%s\n' '--- all shell steps containing workflow expressions ---'
python3 - <<'PY'
from pathlib import Path
p = Path(".github/workflows/translate-docs.yml")
lines = p.read_text().splitlines()
in_run = False
for i, line in enumerate(lines, 1):
if line.lstrip().startswith("run:"):
in_run = True
elif in_run and line and not line.startswith(" "):
in_run = False
if in_run and "${{" in line:
print(f"{i}: {line}")
PY
printf '%s\n' '--- permissions and dispatch-related expressions ---'
rg -n -C 3 'permissions:|workflow_dispatch|inputs\.languages|matrix\.lang|run:' \
.github/workflows/translate-docs.ymlRepository: FailproofAI/failproofai Length of output: 10879 🏁 Script executed: #!/usr/bin/env bash
set -euo pipefail
python3 - <<'PY'
from pathlib import Path
import subprocess
text = Path(".github/workflows/translate-docs.yml").read_text()
supported = {"zh","ja","ko","es","pt-br","de","fr","ru","hi","tr","vi","it","ar","he"}
print("supported language count:", len(supported))
print("allowlist declaration found:", any(
token in text for token in ("case", "allowed", "allowlist", "SUPPORTED_LANGUAGES")
))
# Model the workflow's jq transformation for representative dispatch inputs.
payloads = [
"zh,ja",
"zh,$(printf INJECTED)",
"zh'); printf INJECTED >&2; #",
]
for payload in payloads:
source = f'''if [ -n "{payload}" ]; then
echo "languages=$(echo '{payload}' | jq -Rc 'split(",") | map(gsub("\\\\s"; ""))')" >> "$GITHUB_OUTPUT"
fi
'''
syntax = subprocess.run(["bash", "-n"], input=source, text=True,
capture_output=True)
print(f"payload={payload!r} bash_syntax={syntax.returncode == 0}")
if syntax.stderr:
print("syntax_error:", syntax.stderr.strip())
# Show the generated shell source for matrix.lang at the reviewed step.
matrix_payload = 'zh"; printf INJECTED >&2; #'
warning = (
'echo "::warning title=Translation cache MISS::'
+ matrix_payload
+ ' will re-translate every page (~9 runner-minutes and one full LLM pass)"'
)
print("matrix warning source:", warning)
syntax = subprocess.run(["bash", "-n"], input=warning + "\n", text=True,
capture_output=True)
print("matrix warning syntax valid:", syntax.returncode == 0)
print("matrix warning syntax error:", syntax.stderr.strip() or "<none>")
PYRepository: FailproofAI/failproofai Length of output: 601 Allowlist The workflow has no language allowlist. A dispatch value can inject shell syntax in 🧰 Tools🪛 zizmor (1.29.0)[warning] 115-115: code injection via template expansion (template-injection): may expand into attacker-controllable code (template-injection) 🤖 Prompt for AI AgentsSource: Linters/SAST tools |
||
|
|
||
| - name: Translate ${{ matrix.lang }} | ||
| run: bun run translate --languages ${{ matrix.lang }} ${{ inputs.force == true && '--force' || '' }} | ||
|
|
@@ -99,22 +126,45 @@ jobs: | |
| - name: Validate translated pages parse and images resolve | ||
| run: bun run validate:mdx | ||
|
|
||
| # Save HERE, per language, in the job that produced the work and directly | ||
| # after the step that proved it good. | ||
| # | ||
| # The only save used to be `consolidate`'s, downstream of BOTH the matrix | ||
| # gate (`if: needs.translate.result == 'success'`) and `mintlify validate`. | ||
| # So one page failing validation in one language threw away the cache for | ||
| # all fourteen — Aug 6 lost ~110 minutes of completed translation to a | ||
| # single `ko` page — and a nav mismatch in consolidate did the same on | ||
| # Aug 12. Each fragment is already authoritative for its own language, so | ||
| # there is nothing a merge has to happen first for. | ||
| # | ||
| # The `cache-hit` guard is the same one `build-daemon.yml:137` carries, and | ||
| # it is load-bearing here for a specific reason: the key embeds | ||
| # `github.run_id`, which is REUSED when someone re-runs a failed job. On | ||
| # that second attempt the primary key already exists, so the restore above | ||
| # scores an exact hit and this save would collide with itself. | ||
| - name: Save translation cache fragment | ||
| if: steps.restore-cache.outputs.cache-hit != 'true' | ||
| uses: actions/cache/save@v6 | ||
| with: | ||
| path: scripts/translate-docs/.translation-cache.json | ||
| key: translation-cache-${{ matrix.lang }}-${{ github.run_id }} | ||
|
|
||
| - name: Upload translated files | ||
| uses: actions/upload-artifact@v7 | ||
| with: | ||
| name: translations-${{ matrix.lang }} | ||
| path: | | ||
| docs/${{ matrix.lang }}/ | ||
| docs/i18n/README.${{ matrix.lang }}.md | ||
| retention-days: 1 | ||
| retention-days: 7 | ||
| if-no-files-found: error | ||
|
|
||
| - name: Upload cache fragment | ||
| uses: actions/upload-artifact@v7 | ||
| with: | ||
| name: cache-${{ matrix.lang }} | ||
| path: scripts/translate-docs/.translation-cache.json | ||
| retention-days: 1 | ||
| retention-days: 7 | ||
| if-no-files-found: error | ||
| include-hidden-files: true | ||
|
|
||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
Repository: FailproofAI/failproofai
Length of output: 20013
🏁 Script executed:
Repository: FailproofAI/failproofai
Length of output: 814
Restrict production tags to
main.Manual runs can select any branch or tag ref. With
push_to_ghcr: true, they publishlatest,sha-<short>, and anytag_suffixto GHCR. Restrict publishing torefs/heads/main, or use non-production tags for other refs.🤖 Prompt for AI Agents