Skip to content

feat(assessment): Consolidate assessment_run table - #1237

Draft
vprashrex wants to merge 4 commits into
mainfrom
feat/assessment-consolidation
Draft

vprashrex wants to merge 4 commits into
mainfrom
feat/assessment-consolidation

Conversation

@vprashrex

@vprashrex vprashrex commented Oct 6, 2026 •

Copy link
Copy Markdown
Collaborator

Issue

Closes #PLEASE_TYPE_ISSUE_NUMBER

Summary

  • Before: assessment_run was a 1:N child for multi-config submission, which is no longer supported.
  • Now: The live BATCH path reads and writes assessment directly, with 1:1 relationships in all environments.
  • Migration included adding foreign keys and new fields to assessment.
  • Redundant tables and incomplete records were cleaned up, and unused models were removed.

Checklist

Before submitting a pull request, please ensure that you mark these task.

  • Migration upgrade → downgrade → upgrade exercised on a local DB with seeded pre-fold rows; backfill, error merge, key strip and recreated DDL verified
  • Backfilled JSONB validates against AssessmentExecution
  • All rewired modules import; app.api.main loads; mypy shows 0 new errors (4 fixed)
  • End-to-end BATCH assessment on dev (needs provider credentials)
  • Tests under app/tests/assessment/test_api_*.py updated (separate step)

Notes

Please add here if any other information is required for the reviewer.

Original PR description

Issue

Closes #PLEASE_TYPE_ISSUE_NUMBER

Summary

assessment_run was built as a 1:N child so one submission could fan out to up to four configs. The console no longer offers multi-config submission and the API-client path always creates exactly one execution, so the tables are 1:1 in every environment (prod: 336 assessments, 336 runs, all BATCH, 0 duplicates). The child is redundant.

PR 1 of 3. This PR lands the final schema and the minimal wiring so the live BATCH path reads and writes assessment directly. PR 2 deletes the retired RUN pipeline code; PR 3 is cleanups.

Migration 086

  • assessment gains config_id (FK, NOT NULL), config_version (NOT NULL), total_items (NOT NULL), execution (JSONB, nullable for the future RESPONSE method).
  • Backfilled from assessment_run in one idempotent statement. error_message folds into execution.error; bag keys the code no longer reads (stage_output_urls, provider, model, input_schema) are stripped.
  • assessment.input dropped (RUN-only, never written by BATCH).
  • assessment_run dropped. Downgrade recreates it empty with the original DDL; rows come back from the pg_dump snapshot taken before upgrading.
  • An assessment with no run fails the SET NOT NULL on purpose rather than migrating half.

AssessmentExecution

Pydantic model (extra="forbid", int row keys) replaces the BatchRunState TypedDict. ApiStage / StageKind move into models/assessment/assessment_api.py. stage_kind() / next_stage() live on the model.

Wiring

  • crud/assessment/api.py: create_assessment takes the pinned config, total_items and the seeded bag — one insert. save_execution_state / load_execution_state on Assessment. create_execution, set_execution_batch_job, list_executions, list_assessments_with_execution removed.
  • services/assessment/api/{batch,results,result_files,submission}.py: operate on the assessment row. Provider/model resolved from the pinned config version; stage dump urls read from result_files.
  • Celery run_assessment_api_batch takes assessment_id.
  • runs.py and assessments.py routers unregistered (RUN-only, no callers; modules deleted in PR 2).
  • Wiki assessment.md + domain-map.md updated.

API contract: unchanged. Frontend: no coordination needed (verified against origin/main).

Checklist

  • Migration upgrade → downgrade → upgrade exercised on a local DB with seeded pre-fold rows; backfill, error merge, key strip and recreated DDL verified
  • Backfilled JSONB validates against AssessmentExecution
  • All rewired modules import; app.api.main loads; mypy shows 0 new errors (4 fixed)
  • End-to-end BATCH assessment on dev (needs provider credentials)
  • Tests under app/tests/assessment/test_api_*.py updated (separate step)

Deploy notes

  1. pg_dump -t assessment_run --data-only before migrating. The table is dropped; this is the rollback source.
  2. Drain the Celery queue. A queued pre-fold task carries execution_id, which the new worker does not accept.
  3. Migrate, then deploy code.

Rollback: alembic downgrade -1, restore the dump, redeploy the previous image.

Additive expand step toward folding the 1:1 assessment_run child into
its parent. Adds the seven run-only columns to assessment (nullable, no
table rewrite), backfills them from assessment_run, and leaves
assessment_run untouched so deployed code keeps working. The code
switch and the child-table drop follow in a separate PR.
…ve legacy RUN pipeline

- Merged AssessmentRun into Assessment, simplifying the data model and removing unnecessary complexity.
- Updated AssessmentExecution to encapsulate execution state and related data, replacing BatchRunState.
- Refactored error handling and result file management to streamline error reporting and file uploads.
- Adjusted API routes and documentation to reflect the removal of the RUN method and the focus on BATCH and RESPONSE methods.
- Enhanced type annotations and improved code readability across various service modules.
- Updated submission handling to ensure consistent execution state management and error handling.
@coderabbitai

coderabbitai Bot commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are limited based on label configuration.

🏷️ Required labels (at least one) (1)
  • ready-for-review

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration
  • Configuration used: Repository: ProjectTech4DevAI/kaapi-backend/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: b2381322-e185-4786-90dc-e787073df425

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot changed the title feat(assessment): fold assessment_run into assessment feat(assessment): Consolidate assessment_run data Oct 6, 2026
@github-actions

github-actions Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

OpenAPI changes   🔴 11 breaking changes

Caution

Downstream consumers may need an update before merging.

Breaking changes  ·  11
Method Path Change
🔴 GET /api/v1/assessment/assessments api path removed without deprecation
🔴 GET /api/v1/assessment/assessments/{assessment_id} api path removed without deprecation
🔴 GET /api/v1/assessment/assessments/{assessment_id}/results api path removed without deprecation
🔴 POST /api/v1/assessment/assessments/{assessment_id}/retry api path removed without deprecation
🔴 GET /api/v1/assessment/runs api path removed without deprecation
🔴 POST /api/v1/assessment/runs api path removed without deprecation
🔴 GET /api/v1/assessment/runs/{run_id} api path removed without deprecation
🔴 PATCH /api/v1/assessment/runs/{run_id}/post-processing api path removed without deprecation
🔴 GET /api/v1/assessment/runs/{run_id}/results api path removed without deprecation
🔴 POST /api/v1/assessment/runs/{run_id}/resume api path removed without deprecation
🔴 POST /api/v1/assessment/runs/{run_id}/retry api path removed without deprecation
Full changelog  ·  33
Method Path Change
🔴 GET /api/v1/assessment/assessments api path removed without deprecation
🔴 GET /api/v1/assessment/assessments/{assessment_id} api path removed without deprecation
🔴 GET /api/v1/assessment/assessments/{assessment_id}/results api path removed without deprecation
🔴 POST /api/v1/assessment/assessments/{assessment_id}/retry api path removed without deprecation
🔴 GET /api/v1/assessment/runs api path removed without deprecation
🔴 POST /api/v1/assessment/runs api path removed without deprecation
🔴 GET /api/v1/assessment/runs/{run_id} api path removed without deprecation
🔴 PATCH /api/v1/assessment/runs/{run_id}/post-processing api path removed without deprecation
🔴 GET /api/v1/assessment/runs/{run_id}/results api path removed without deprecation
🔴 POST /api/v1/assessment/runs/{run_id}/resume api path removed without deprecation
🔴 POST /api/v1/assessment/runs/{run_id}/retry api path removed without deprecation
🟢 — — removed the schema APIResponse_AssessmentPublic_
🟢 — — removed the schema APIResponse_AssessmentResponse_
🟢 — — removed the schema APIResponse_AssessmentRunPublic_
🟢 — — removed the schema APIResponse_AssessmentRunResponse_
🟢 — — removed the schema APIResponse_list_AssessmentPublic__
🟢 — — removed the schema APIResponse_list_AssessmentRunPublic__
🟢 — — removed the schema APIResponse_list_dict_str__Any___
🟢 — — removed the schema AssessmentAttachment
🟢 — — removed the schema AssessmentExecutionPublic
🟢 — — removed the schema AssessmentPublic
🟢 — — removed the schema AssessmentResponse
🟢 — — removed the schema AssessmentRunCounts
🟢 — — removed the schema AssessmentRunCreate
🟢 — — removed the schema AssessmentRunPublic
🟢 — — removed the schema AssessmentRunResponse
🟢 — — removed the schema AssessmentRunStat
🟢 — — removed the schema AssessmentRunSummary
🟢 — — removed the schema InputBinding
🟢 — — removed the schema RunExecution
🟢 — — removed the schema Stage
🟢 — — removed the schema StageStatus
🟢 — — a breaking change was detected but the version is still 0.5.0

main ↔ f353c251 · generated by oasdiff

@vprashrex vprashrex changed the title feat(assessment): Consolidate assessment_run data feat(assessment): Consolidate assessment_run table Oct 10, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant