[PowerX] persist telemetry and recover historical reads / 持久化遥测并恢复历史数据读取

.github/workflows/apply-run-overrides.yml

@@ -53,7 +53,18 @@ jobs:
         env:
           CYPRESS_INSTALL_BINARY: '0'
 
+      # This workflow fires on a push to run-overrides.ts, which can land before the
+      # next ingest dispatch has applied a pending migration. admin:db:verify counts
+      # every table it knows about, so without this step it fails on a schema the
+      # checked-out ref expects but production does not have yet. Same command and
+      # ordering as ingest-results.yml; migrations are idempotent by filename.
+      - name: Run migrations
+        env:
+          DATABASE_WRITE_URL: ${{ secrets.DATABASE_WRITE_URL }}
+        run: bun run admin:db:migrate --yes
+
       - name: Apply run overrides
+        id: apply
         env:
           DATABASE_WRITE_URL: ${{ secrets.DATABASE_WRITE_URL }}
           RUN_ID: ${{ inputs.run_id }}
@@ -69,10 +80,15 @@ jobs:
           DATABASE_WRITE_URL: ${{ secrets.DATABASE_WRITE_URL }}
         run: bun run admin:db:verify
 
+      # Once the overrides are in the database the CDN is stale, so these must run
+      # even if the verify step above fails. Skipping them leaves the dashboard
+      # serving pre-override data with only a red workflow as the signal.
       - name: Invalidate production cache
+        if: ${{ always() && steps.apply.outcome == 'success' }}
         env:
           INVALIDATE_SECRET: ${{ secrets.VERCEL_INVALIDATE_SECRET }}
         run: bun run admin:cache:invalidate https://inferencex.semianalysis.com
 
       - name: Warm production cache
+        if: ${{ always() && steps.apply.outcome == 'success' }}
         run: bun run admin:cache:warmup https://inferencex.semianalysis.com

.github/workflows/ingest-agentic-results.yml

@@ -31,6 +31,11 @@ on:
         description: InferenceX Actions run ID to ingest
         required: true
         type: string
+      require-power:
+        description: Require the versioned PowerX manifest and all declared evidence
+        required: false
+        default: false
+        type: boolean
       run-attempt:
         description: InferenceX Actions run attempt to ingest
         required: false
@@ -85,6 +90,11 @@ on:
         description: InferenceX Actions run ID to ingest
         required: true
         type: string
+      require-power:
+        description: Require the versioned PowerX manifest and all declared evidence
+        required: false
+        default: false
+        type: boolean
       run-attempt:
         description: InferenceX Actions run attempt to ingest
         required: false
@@ -276,6 +286,7 @@ jobs:
             github.event.client_payload.run-id || inputs.run-id }}
           ARTIFACTS_PATH: ${{ github.workspace }}/artifacts
           INGEST_REPO: SemiAnalysisAI/InferenceX
+          INGEST_REQUIRE_POWER: ${{ github.event.client_payload.require-power == true || inputs.require-power == true }}
         run: bun run admin:db:prepare:ci
 
       - name: Run migrations
@@ -288,6 +299,7 @@ jobs:
           INGEST_RUN_ATTEMPT: ${{ steps.artifacts.outputs.merge-run-attempt }}
           INGEST_ARTIFACTS_PATH: ${{ github.workspace }}/artifacts
           INGEST_REPO: SemiAnalysisAI/InferenceX
+          INGEST_REQUIRE_POWER: ${{ github.event.client_payload.require-power == true || inputs.require-power == true }}
           UNMAPPED_ENTITIES_OUTPUT: ${{ github.workspace }}/unmapped-entities.json
           POWER_PUBLICATION_MANIFEST: ${{ github.workspace }}/power-publication.json
         run: bun run admin:db:ingest:ci

.github/workflows/ingest-results.yml

@@ -60,6 +60,7 @@ jobs:
             github.event.client_payload.run-id }}
           ARTIFACTS_PATH: ${{ github.workspace }}/artifacts
           INGEST_REPO: SemiAnalysisAI/InferenceX
+          INGEST_REQUIRE_POWER: ${{ github.event.client_payload.require-power == true }}
         run: bun run admin:db:prepare:ci
 
       - name: Run migrations
@@ -75,6 +76,7 @@ jobs:
           INGEST_RUN_ATTEMPT: ${{ steps.artifacts.outputs.merge-run-attempt }}
           INGEST_ARTIFACTS_PATH: ${{ github.workspace }}/artifacts
           INGEST_REPO: SemiAnalysisAI/InferenceX
+          INGEST_REQUIRE_POWER: ${{ github.event.client_payload.require-power == true }}
           POWER_PUBLICATION_MANIFEST: ${{ github.workspace }}/power-publication.json
           UNMAPPED_ENTITIES_OUTPUT: ${{ github.workspace }}/unmapped-entities.json
         run: bun run admin:db:ingest:ci

AGENTS.md

@@ -85,6 +85,7 @@ API routes (`packages/app/src/app/api/v1/`):
 - `reliability` — raw `ReliabilityRow[]`
 - `evaluations` — raw `EvalRow[]`
 - `server-log` — retrieve benchmark runtime logs
+- `gpu-metrics-point?id=N` — PowerX chip telemetry (samples + per-GPU digest) linked to one benchmark point
 - `invalidate` — invalidate API cache (admin; `?scope=collectivex` purges only that scope)
 - `collectivex/latest`, `collectivex/runs`, `collectivex/runs/[runId]` — CollectiveX sweep data
   from a **separate** Neon DB, populated lazily on read from GitHub Actions artifacts and served

docs/dashboard-readonly-views.md

@@ -44,7 +44,9 @@ combinations, not the full Cartesian product of all possible filter values.
 - Evaluation/reliability: chart-data, date resolution and rolling aggregation.
 - CollectiveX: selected EP/KV/swap chart and fit helpers.
 - Submissions/images: existing table, weekly/cumulative and image freshness helpers.
-- GPU metrics: shared line/correlation transforms and unsampled statistics.
+- GPU metrics: shared line/correlation transforms and stored full-record per-GPU
+  digests. Live artifacts alone calculate statistics from samples; empty stored
+  digests remain empty. File/host identity and missing-versus-zero semantics persist.
 - Video: checksum-verified stored bundles, serving/fidelity selectors and tradeoffs.
 - Overview/rankings/compare: existing discovery-page assembly and scenario helpers.
 
@@ -61,6 +63,11 @@ OperatorX is feature-gated in navigation and uses page-owned
 contract.
 Zoom, theme, axis scale, labels, media playback and report expansion are renderer
 state. GPU interactive downsampling does not alter returned raw data or statistics.
+The GPU statistics table includes startup and warmup for all chips in the selected
+series, regardless of chip visibility. It is separate from serving-window power,
+J/token and selected-time-window calculations. Run telemetry is DB-first with an
+artifact fallback for missing storage; the public view returns private, no-store
+responses and preserves upstream 503 failures.
 
 Run-specific recognition labels are also presentation-only. Run `35879254139`
 displays `UMBP MoRI SGLang` through October 9, 2026 in America/New_York
@@ -89,6 +96,10 @@ those properties.
 私有上传、密钥、提示词、反馈及管理操作不作为公开读取接口。
 OperatorX 的入口受功能开关控制，页面使用专属的 `/api/v1/operatorx/*`
 接口；目前没有发布 `/api/v1/views/operatorx` 契约。
+GPU 视图优先读取已存遥测，缺少存储数据时回退到产物。全记录统计使用所选文件、
+主机序列的全部芯片摘要，包含启动与 warmup；已有摘要为空时不补算，缺失读数不补零。
+芯片显隐和图表降采样不改变该统计，也不改变 serving-window 或 J/token 的计算口径。
+响应使用 private, no-store，上游 503 保留为错误响应。
 
 测试覆盖契约同步及代表性的筛选行为，并未穷举所有参数组合。生产数据库上的
 完整 UI/API 对照仍需集成审查，不能仅凭单元测试宣称已完成。