fix(prism): AutoModel pin mount + in-flight site submissions - #128
Conversation
Prod intake was fail-closed (code=pin) without a mounted pin tree, and the site submissions gallery hid everything except Score>0 champions — so the AutoModel FE tab looked empty even while miners were blocked. Mount the staged pin on master compose, default /submissions to scope=all with legacy era fallback, and warn remote-deploy when the pin directory is missing.
|
Warning Review limit reached
Next review available in: 48 minutes You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughPrism submissions now support ChangesPrism API behavior
AutoModel pin deployment
Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant Client
participant SiteAPI
participant PrismAPI
Client->>SiteAPI: Request submissions with optional scope
SiteAPI->>PrismAPI: Fetch submission listings
PrismAPI-->>SiteAPI: Return listings and detail data
SiteAPI-->>Client: Return scoped submissions with recipeEra
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches 💡 1🛠️ Fix failing CI checks 💡
📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Pick up /var/lib/prism/docker-compose.automodel-pin.yml on master so a redeploy before env-prod bake-in cannot drop the pin mount again.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🧹 Nitpick comments (3)
crates/site-api/src/handlers.rs (3)
1225-1241: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winCover failed and zero-score rows in the scope test.
The new fixture covers a running row only. The documented contract also promises that
scope=allincludes failed and Score=0 rows, whilescope=championsexcludes them. Add a failed, zero-score fixture and assert both scope results.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/site-api/src/handlers.rs` around lines 1225 - 1241, Add a failed submission fixture with a zero score to the scope test setup, then update the scope=all assertions to verify it is included and the total reflects it. Update the scope=champions assertions to verify the failed zero-score row is excluded while the existing positive-score row remains included.
453-456: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAdd regression coverage for the leaderboard fallback.
The current Prism leaderboard fixture resolves an AutoModel era for
sub1. It does not exercise the newrow.recipe_era.is_none()branch. Add a champion whose detail request fails and assert thatrecipeErais"legacy".🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/site-api/src/handlers.rs` around lines 453 - 456, Add regression coverage for the fallback in the leaderboard handler by extending the existing Prism leaderboard fixture with a champion whose detail request fails, leaving row.recipe_era unset. Assert that the resulting champion’s recipeEra is "legacy", while preserving the existing AutoModel-era assertions for other champions.
554-560: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick winReject unsupported
scopevalues.Only
allandchampionsare documented, but every value other thanchampionsorchampioncurrently selectsall. A typo such asscope=championsssilently broadens the result set. Parse supported values explicitly and return a client error for unsupported values.Proposed validation
- let champions_only = matches!(scope.as_str(), "champions" | "champion"); + let champions_only = match scope.as_str() { + "all" => false, + "champions" | "champion" => true, + _ => { + return json_err( + StatusCode::BAD_REQUEST, + "invalid_scope", + "scope must be all or champions", + ); + } + };🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/site-api/src/handlers.rs` around lines 554 - 560, Update the scope parsing in the handler around scope and champions_only to accept only “all” and “champions” (optionally preserving the existing “champion” alias if intended), and return an appropriate client error for any other value instead of defaulting to all. Preserve the existing champions_only behavior for supported values.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/site-api/src/handlers.rs`:
- Around line 566-569: Update the detail fan-out logic around
fetch_prism_details to filter items by status_filter before limiting the IDs,
then order the eligible rows with in-flight statuses prioritized while
preserving recency within each group. Truncate only after prioritization, and
add a regression test covering pending or failed rows beyond the initial 24
receiving detail enrichment instead of RecipeEra::Legacy.
In `@deploy/compose/env-staging.yml`:
- Around line 63-68: Update the staging service configuration around
PRISM_FORCE_SIM and PRISM_AUTOMODEL_PIN_DIR so staging does not run the Sim path
or use Sim/fixture settings. Set PRISM_FORCE_SIM to false and configure
challenge execution through the Docker-backed path, keeping SimSandbox and
BASE_ALLOW_HOST_SIM opt-ins limited to CI/local configuration while preserving
the live AutoModel pin mount for staging.
In `@deploy/scripts/remote-deploy.sh`:
- Around line 263-269: Update the AutoModel pin check in the remote deployment
validation block to run the existing stage-automodel-pin.sh --verify-only
validation instead of only testing for the .git directory. Report success only
when the verification passes; otherwise retain the warning guidance for staging
the pin.
- Around line 270-275: Update the deployment validation around the AutoModel
overlay inclusion in the remote-deploy flow to invoke assert-compose-matrix.sh
against rendered master staging and production stacks with
/var/lib/prism/docker-compose.automodel-pin.yml applied. Ensure validation
rejects changes to role constraints, challenge environment files, the read-only
AutoModel pin mount, and digest-pinned images before adding the overlay to
COMPOSE_FILES.
---
Nitpick comments:
In `@crates/site-api/src/handlers.rs`:
- Around line 1225-1241: Add a failed submission fixture with a zero score to
the scope test setup, then update the scope=all assertions to verify it is
included and the total reflects it. Update the scope=champions assertions to
verify the failed zero-score row is excluded while the existing positive-score
row remains included.
- Around line 453-456: Add regression coverage for the fallback in the
leaderboard handler by extending the existing Prism leaderboard fixture with a
champion whose detail request fails, leaving row.recipe_era unset. Assert that
the resulting champion’s recipeEra is "legacy", while preserving the existing
AutoModel-era assertions for other champions.
- Around line 554-560: Update the scope parsing in the handler around scope and
champions_only to accept only “all” and “champions” (optionally preserving the
existing “champion” alias if intended), and return an appropriate client error
for any other value instead of defaulting to all. Preserve the existing
champions_only behavior for supported values.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: cd5b8d6e-5d9b-4e61-b1a4-e012d50ed49c
📒 Files selected for processing (6)
crates/site-api/src/handlers.rsdeploy/compose/env-prod.ymldeploy/compose/env-staging.ymldeploy/scripts/remote-deploy.shdocs/PRISM_RECIPE.mddocs/SITE_API.md
| // Prefer in-flight + recent rows for detail fan-out (era / benches). | ||
| let mut ids: Vec<String> = items.iter().map(|s| s.id.clone()).collect(); | ||
| ids.truncate(PRISM_CHAMPION_DETAIL_FANOUT); | ||
| let details = fetch_prism_details(&st, &ids).await; |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
rg -n -C 5 \
'fetch_prism_subs|PRISM_CHAMPION_DETAIL_FANOUT|updated_at_ms|created_at_ms' \
--glob '*.rs' .Repository: BaseIntelligence/base
Length of output: 50378
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo '--- handlers structure ---'
ast-grep outline crates/site-api/src/handlers.rs | sed -n '1,220p'
echo '--- relevant handler references ---'
rg -n -C 12 \
'PRISM_CHAMPION_DETAIL_FANOUT|fetch_prism_subs|fetch_prism_details|scope|list_submissions|submissions' \
crates/site-api/src/handlers.rs crates/site-api/src --glob '*.rs' | sed -n '1,320p'
echo '--- store APIs and implementations ---'
rg -n -C 10 \
'async fn (list|fetch).*submission|fn (list|fetch).*submission|list_submissions|list_recent|submissions.*ORDER BY|ORDER BY.*(created|updated)' \
crates --glob '*.rs' | sed -n '1,360p'Repository: BaseIntelligence/base
Length of output: 49030
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo '--- Prism submissions handler and leaderboard ---'
sed -n '421,590p' crates/site-api/src/handlers.rs
echo '--- Prism store list definitions ---'
rg -n -C 18 \
'pub async fn list\(|async fn list\(|ORDER BY.*created_at|ORDER BY.*updated_at|prism_submission' \
crates/prism-store crates/prism-challenge/src --glob '*.rs' | sed -n '1,420p'
echo '--- repository files related to prism submission storage ---'
git ls-files crates/prism-store crates/prism-challenge/src | grep -E '(submission|store|lib|api)' | sed -n '1,200p'Repository: BaseIntelligence/base
Length of output: 35564
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo '--- MemoryPrismStore ordering ---'
sed -n '340,410p' crates/prism-store/src/store.rs
echo '--- DbPrismStore list implementation ---'
rg -n -C 24 \
'async fn list\(' crates/prism-store/src/dbprism.rs
echo '--- database list query ---'
rg -n -C 24 \
'list_prism_submissions|prism_submissions.*ORDER|FROM prism_submission.*ORDER|ORDER BY.*created_at.*prism' \
crates --glob '*.rs' | sed -n '1,260p'
echo '--- relevant tests and helper fixtures ---'
rg -n -C 10 \
'mount_prism_list_mocks|prism_leaderboard|scope=|champions|sub-running|PRISM_CHAMPION_DETAIL_FANOUT|detail' \
crates/site-api/src/handlers.rs | sed -n '1,420p'Repository: BaseIntelligence/base
Length of output: 27431
🏁 Script executed:
#!/bin/bash
set -euo pipefail
python3 - <<'PY'
from pathlib import Path
handler = Path("crates/site-api/src/handlers.rs").read_text()
db = Path("crates/db/src/prism_store.rs").read_text()
assert 'ORDER BY created_at DESC LIMIT $3' in db
fanout = handler.index('let mut ids: Vec<String> = items.iter().map(|s| s.id.clone()).collect();')
truncate = handler.index('ids.truncate(PRISM_CHAMPION_DETAIL_FANOUT);', fanout)
status_filter = handler.index('if let Some(st_f) = status_filter {', truncate)
rows = [
{"id": "old-pending", "created_at_ms": 1},
{"id": "new-scored", "created_at_ms": 3},
{"id": "newer-failed", "created_at_ms": 4},
]
ordered = sorted(rows, key=lambda row: row["created_at_ms"], reverse=True)
selected = [row["id"] for row in ordered[:2]]
assert selected == ["newer-failed", "new-scored"]
assert "old-pending" not in selected
assert truncate < status_filter
print("list_prism_submissions guarantees created_at DESC, not in-flight-first ordering")
print("the handler truncates that order before applying status_filter")
print("synthetic order:", [row["id"] for row in ordered])
print("fan-out IDs:", selected)
PYRepository: BaseIntelligence/base
Length of output: 409
Prioritize in-flight rows before truncating detail fan-out. /v1/submissions guarantees created_at DESC, but not in-flight-first ordering. This code truncates to 24 rows before applying status_filter, so older pending or failed rows can miss detail enrichment and fall back to RecipeEra::Legacy. Apply the filter before truncation and prioritize in-flight rows. Add a regression test.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@crates/site-api/src/handlers.rs` around lines 566 - 569, Update the detail
fan-out logic around fetch_prism_details to filter items by status_filter before
limiting the IDs, then order the eligible rows with in-flight statuses
prioritized while preserving recency within each group. Truncate only after
prioritization, and add a regression test covering pending or failed rows beyond
the initial 24 receiving detail enrichment instead of RecipeEra::Legacy.
| # Same pin path as prod when testing live AutoModel intake on staging | ||
| # (stage with deploy/scripts/stage-automodel-pin.sh). Sim/fixture pins | ||
| # still work when miners submit automodel@fixture-v1. | ||
| PRISM_AUTOMODEL_PIN_DIR: "/var/lib/prism/automodel-pin" | ||
| volumes: | ||
| - /var/lib/prism/automodel-pin:/var/lib/prism/automodel-pin:ro |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift
Do not run the Sim path in staging.
PRISM_FORCE_SIM remains "true" for this service at Line 60. The new mount cannot provide a live AutoModel execution check while Sim is forced. Move the Sim and fixture configuration to CI/local, or set PRISM_FORCE_SIM to "false" and use the Docker-backed challenge path.
As per coding guidelines, deploy/**/* must never host Sim in staging or production; SimSandbox and BASE_ALLOW_HOST_SIM=1 are CI/local-only opt-ins. Based on learnings, staging challenge execution must use Docker and must not host Sim.
Suggested configuration change
- PRISM_FORCE_SIM: "true"
+ PRISM_FORCE_SIM: "false"🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@deploy/compose/env-staging.yml` around lines 63 - 68, Update the staging
service configuration around PRISM_FORCE_SIM and PRISM_AUTOMODEL_PIN_DIR so
staging does not run the Sim path or use Sim/fixture settings. Set
PRISM_FORCE_SIM to false and configure challenge execution through the
Docker-backed path, keeping SimSandbox and BASE_ALLOW_HOST_SIM opt-ins limited
to CI/local configuration while preserving the live AutoModel pin mount for
staging.
Sources: Coding guidelines, Learnings
| if ssh_h "test -d /var/lib/prism/automodel-pin/.git"; then | ||
| echo "remote-deploy: AutoModel pin present at /var/lib/prism/automodel-pin" | ||
| else | ||
| echo "remote-deploy: WARNING: AutoModel pin missing at /var/lib/prism/automodel-pin" >&2 | ||
| echo "remote-deploy: stage with: ./deploy/scripts/stage-automodel-pin.sh --dir /var/lib/prism/automodel-pin" >&2 | ||
| echo "remote-deploy: (Prism AutoModel intake fails closed with code=pin until staged)" >&2 | ||
| fi |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Validate the pin contents before reporting success.
test -d /var/lib/prism/automodel-pin/.git accepts a stale or modified checkout. deploy/scripts/stage-automodel-pin.sh already verifies the frozen commit and content SHA. Reuse its --verify-only validation, or compare both expected values here. Otherwise, deployment can report the pin as present while Prism intake still returns code=pin.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@deploy/scripts/remote-deploy.sh` around lines 263 - 269, Update the AutoModel
pin check in the remote deployment validation block to run the existing
stage-automodel-pin.sh --verify-only validation instead of only testing for the
.git directory. Report success only when the verification passes; otherwise
retain the warning guidance for staging the pin.
| if ssh_h "test -f /var/lib/prism/docker-compose.automodel-pin.yml"; then | ||
| COMPOSE_FILES+=(-f /var/lib/prism/docker-compose.automodel-pin.yml) | ||
| echo "remote-deploy: including host AutoModel pin overlay" | ||
| fi | ||
| fi | ||
|
|
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
./deploy/scripts/assert-compose-matrix.shRepository: BaseIntelligence/base
Length of output: 375
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- deploy/scripts/remote-deploy.sh ---'
sed -n '220,290p' deploy/scripts/remote-deploy.sh
printf '%s\n' '--- deploy/scripts/assert-compose-matrix.sh ---'
sed -n '1,240p' deploy/scripts/assert-compose-matrix.sh
printf '%s\n' '--- compose files and relevant references ---'
git ls-files 'deploy/compose/*' 'deploy/scripts/*' 'deploy/env/*' | sort
rg -n --no-heading 'automodel|challenge|digest|profile:|gateway|validator|postgres|BASE_ALLOW_HOST_SIM' \
deploy/compose deploy/scripts deploy/env 2>/dev/null | head -300Repository: BaseIntelligence/base
Length of output: 45901
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- remote deployment Compose invocation ---'
sed -n '290,430p' deploy/scripts/remote-deploy.sh
printf '%s\n' '--- base and role Compose definitions ---'
sed -n '1,260p' docker-compose.yml
sed -n '1,180p' deploy/compose/role-master.yml
sed -n '1,130p' deploy/compose/env-staging.yml
sed -n '1,120p' deploy/compose/env-prod.yml
printf '%s\n' '--- AutoModel staging script ---'
sed -n '1,150p' deploy/scripts/stage-automodel-pin.sh
printf '%s\n' '--- overlay references ---'
rg -n --no-heading 'docker-compose\.automodel-pin|COMPOSE_FILES|config --|docker compose' deploy/scripts/remote-deploy.sh deploy/scripts/stage-automodel-pin.shRepository: BaseIntelligence/base
Length of output: 29532
Validate the host AutoModel overlay before including it.
./deploy/scripts/assert-compose-matrix.sh does not load /var/lib/prism/docker-compose.automodel-pin.yml. Extend the deployment check to render the master staging and production stacks with the overlay and reject changes to role constraints, challenge environment files, the read-only AutoModel pin mount, or digest-pinned images.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@deploy/scripts/remote-deploy.sh` around lines 270 - 275, Update the
deployment validation around the AutoModel overlay inclusion in the
remote-deploy flow to invoke assert-compose-matrix.sh against rendered master
staging and production stacks with
/var/lib/prism/docker-compose.automodel-pin.yml applied. Ensure validation
rejects changes to role constraints, challenge environment files, the read-only
AutoModel pin mount, and digest-pinned images before adding the overlay to
COMPOSE_FILES.
Source: Coding guidelines
1 participant
Summary
/var/lib/prism/automodel-pinintoprism-challengeon prod/staging compose and setPRISM_AUTOMODEL_PIN_DIRso AutoModel 2.0 intake no longer fail-closes withcode=pinafter redeploy.GET /v1/site/arenas/prism/submissionsdefault toscope=all(in-flight + failed + scored); keep?scope=championsfor the Score>0 gallery. Leaderboard stays champions-only.recipeEratolegacyafter detail fan-out so Legacy/All tabs are not empty when enrichment is thin.remote-deploywarns when the staged pin tree is missing on master.Ops already done on prod
/var/lib/prism/automodel-pin(commitd02f49cb…, content sha matches freeze).prism-challengewith pin mount; intake now reaches metagraph/auth checks instead ofcode=pin.Test plan
cargo test -p site-api --libGET /v1/site/arenas/prism/submissionsincludesstatus=pendingrows andrecipeEraGET …/submissions?scope=championsstill Score>0 onlyremote-deploy --env prod --role master, pin warning is silent when/var/lib/prism/automodel-pinexistsSummary by CodeRabbit