Skip to content

test(api): prevent silent v1 OpenAPI breaking changes - #381

Open
seonghobae wants to merge 8 commits into
mainfrom
feat/openapi-v1-compatibility-gate-main-20260811
Open

test(api): prevent silent v1 OpenAPI breaking changes#381
seonghobae wants to merge 8 commits into
mainfrom
feat/openapi-v1-compatibility-gate-main-20260811

Conversation

@seonghobae

@seonghobae seonghobae commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

Objective

Advance issue #315 with a deterministic v1 compatibility gate so repository-owned OpenAPI changes cannot silently remove an existing operation, rename its operationId, or remove an advertised response status without an explicit version/baseline decision.

Exact current-head evidence

  • protected base at latest head construction: 961f44a19c4ced87f8062b3ab9842a9a08d05235;
  • exact head: d5407a24c646abf08f41c1ebb393687795599747;
  • changed paths: offline compatibility checker, immutable v1 baseline manifest, and focused standard-library tests only;
  • CI, Security Scan, and SAST Semgrep: terminal-success;
  • fuzz is not emitted for this Python contract-only path and is not represented as passing evidence;
  • mergeable: true;
  • all inline findings are resolved or outdated.

Compatibility contract

The checker parses the repository-owned OpenAPI structure, normalizes actual HTTP operations, and compares path/method, stable operationId, and advertised response-status sets against the checked-in v1 baseline. Removal, identity change, or response-status removal fails closed. Additive operations and response statuses remain compatible. Formatting variants such as quoted paths, inline comments, and supported anchored response mappings are covered by regression tests.

Scope boundary

Offline compatibility checker and baseline only. This does not change the public OpenAPI, controllers, DTOs, runtime negotiation, generated clients, naruon integration, or release schema provenance. Issue #315 remains open for those guarantees.

Merge gate

Keep this exact head unchanged. Auto-merge may act only after live required checks remain terminal-success, zero valid unresolved findings remain, and a qualifying independent non-author approval is attached to this exact head. Any head/base movement requires complete exact-head revalidation; predecessor evidence does not transfer.

@coderabbitai

coderabbitai Bot commented Aug 11, 2026

Copy link
Copy Markdown

Review Change Stack

Important

Review available on request

  • 🔍 Trigger review

Reviews should be triggered manually for repositories with fewer than 10 stars. Select Trigger review above or comment @coderabbitai review to review the latest changes. For a full review, comment @coderabbitai full review.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: be668c1c-649d-4e92-9dd7-aa3757062fdc

📝 Walkthrough

Walkthrough

OpenAPI v1 호환성 기준선에 API 작업 정보를 추가했습니다. YAML 명세를 수집하고 작업, operationId, 응답 상태 코드의 삭제 및 변경을 검사하는 스크립트와 회귀 테스트를 추가했습니다.

Changes

OpenAPI v1 호환성

Layer / File(s) Summary
기준선 및 호환성 판정 로직
scripts/openapi_v1_compatibility_baseline.json, scripts/openapi_v1_compatibility_contract.py
분석, 변환 작업, 뷰어, 아티팩트 링크 API의 작업과 응답 상태 코드를 기준선에 추가했습니다. YAML에서 작업 정보를 수집하고, 크기 제한과 호환성 위반 판정을 구현했습니다.
호환성 회귀 테스트
scripts/test_openapi_v1_compatibility_contract.py
작업 정보 수집, 작업 삭제, operationId 변경, 응답 삭제를 검증합니다. 응답 및 작업 추가와 저장된 기준선의 유지도 검증합니다.

Estimated code review effort: 3 (Moderate) | ~20 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed 제목은 v1 OpenAPI 호환성 검사를 추가하여 호환성 중단을 방지하는 주요 변경 사항을 명확하게 설명합니다.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/openapi-v1-compatibility-gate-main-20260811

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@seonghobae
seonghobae marked this pull request as ready for review August 11, 2026 08:05

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@scripts/openapi_v1_compatibility_contract.py`:
- Around line 32-82: Update collect_operations to parse the OpenAPI document
structurally as YAML instead of relying on exact indentation and raw string
formats, while preserving extraction of HTTP method, path, operationId, and
response keys. Ensure valid quoted paths, inline operationId comments, and
anchored responses values are handled correctly, and add regression tests
covering these formatting variants.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 6994b3cc-c4ea-4703-bcde-83d6b8084ae0

📥 Commits

Reviewing files that changed from the base of the PR and between 55d7ae8 and e9f8b10.

📒 Files selected for processing (3)
  • scripts/openapi_v1_compatibility_baseline.json
  • scripts/openapi_v1_compatibility_contract.py
  • scripts/test_openapi_v1_compatibility_contract.py

Comment thread scripts/openapi_v1_compatibility_contract.py Outdated

Copy link
Copy Markdown
Collaborator Author

@opencode-agent review
@coderabbitai review
@cwl-noema-review review
@strix review

Review exact head d5407a24c646abf08f41c1ebb393687795599747 only. Exact-head CI, Security Scan, and SAST Semgrep are terminal-success; fuzz is not emitted for this Python contract-only path; all inline findings are resolved or outdated. Validate structural operation extraction, immutable v1 baseline comparison, and fail-closed removal/operationId/response-status compatibility checks while allowing additive changes.

Review read-only; preserve independent reviewer identity. Do not mutate the branch, synthesize author approval, weaken protection, or merge.

@coderabbitai

coderabbitai Bot commented Aug 15, 2026

Copy link
Copy Markdown

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '== HEAD =='
git rev-parse HEAD
printf '%s\n' '== STATUS =='
git status --short
printf '%s\n' '== CHANGED FILES AGAINST FIRST PARENT =='
git diff --name-status HEAD^ HEAD
printf '%s\n' '== DIFF =='
git diff --no-ext-diff --unified=80 HEAD^ HEAD -- \
  scripts/openapi_v1_compatibility_contract.py \
  scripts/openapi_v1_compatibility_baseline.json \
  scripts/test_openapi_v1_compatibility_contract.py
printf '%s\n' '== PYTHON OUTLINE =='
ast-grep outline scripts/openapi_v1_compatibility_contract.py --items all --view expanded
printf '%s\n' '== RELATED WORKFLOW OR SCRIPT REFERENCES =='
rg -n -C 3 --glob '!scripts/openapi_v1_compatibility_contract.py' --glob '!scripts/test_openapi_v1_compatibility_contract.py' \
  'openapi_v1_compatibility_contract|openapi_v1_compatibility_baseline|compatibility contract' .

Length of output: 2404


🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '== PR METADATA =='
gh pr view 381 --repo ContextualWisdomLab/clearfolio \
  --json number,baseRefName,baseRefOid,headRefName,headRefOid,commits,files \
  --jq '{number, baseRefName, baseRefOid, headRefName, headRefOid, commits: [.commits[] | {oid:.oid,messageHeadline:.messageHeadline}], files: [.files[].path]}'

printf '%s\n' '== COMPATIBILITY-GATE COMMITS REACHABLE FROM HEAD =='
git log --oneline --decorate --all -- \
  scripts/openapi_v1_compatibility_contract.py \
  scripts/openapi_v1_compatibility_baseline.json \
  scripts/test_openapi_v1_compatibility_contract.py

printf '%s\n' '== CHECKER =='
cat -n scripts/openapi_v1_compatibility_contract.py
printf '%s\n' '== BASELINE =='
cat -n scripts/openapi_v1_compatibility_baseline.json
printf '%s\n' '== TESTS =='
cat -n scripts/test_openapi_v1_compatibility_contract.py

printf '%s\n' '== OPENAPI CANDIDATE FILES =='
fd -HI -t f -i 'openapi|swagger' . | sed -n '1,160p'

Length of output: 16785


🏁 Script executed:

#!/bin/bash
set -euo pipefail

base='961f44a19c4ced87f8062b3ab9842a9a08d05235'
head='d5407a24c646abf08f41c1ebb393687795599747'

printf '%s\n' '== PR DIFF AGAINST MAIN =='
git diff --no-ext-diff --unified=40 "$base" "$head" -- \
  scripts/openapi_v1_compatibility_contract.py \
  scripts/openapi_v1_compatibility_baseline.json \
  scripts/test_openapi_v1_compatibility_contract.py

printf '%s\n' '== PYTHON TEST AND CI CONFIGURATION =='
fd -HI -t f 'pyproject.toml|pytest.ini|tox.ini|setup.cfg|Pipfile|requirements.*|.*\.ya?ml' . \
  | grep -E '(^|/)(pyproject\.toml|pytest\.ini|tox\.ini|setup\.cfg|Pipfile|requirements[^/]*|\.github/workflows/[^/]+\.ya?ml)$' \
  | while IFS= read -r f; do
      printf '\n--- %s ---\n' "$f"
      rg -n -C 4 -i 'pytest|python|scripts/test|openapi_v1|test_.*\.py' "$f" || true
    done

printf '%s\n' '== OPENAPI PATHS STRUCTURE AND COMPATIBILITY OPERATIONS =='
rg -n -C 3 \
  '^(paths:|  /api/v1/(analytics/kpi-snapshot|analytics/kpi-snapshot-exports|convert/jobs|viewer/)|    (get|post):|      operationId:|      responses:|        (200|202|400|401|403|404|409):)' \
  docs/deployment/clearfolio-buyer-connector.openapi.yaml

printf '%s\n' '== BASELINE MODIFICATION GUARDS =='
rg -n -i -C 3 \
  'openapi_v1_compatibility_baseline|compatibility baseline|baseline.*(immutable|protected|hash|checksum)|immutable.*baseline' \
  .github scripts README.md docs pyproject.toml pytest.ini tox.ini setup.cfg 2>/dev/null || true

Length of output: 25307


@seonghobae I reviewed exact head d5407a24c646abf08f41c1ebb393687795599747.

I found one blocking issue.

  • Baseline is not immutable. scripts/test_openapi_v1_compatibility_contract.py, Line 124, loads the baseline from the same pull request. A pull request can remove a baseline operation, change its operationId, or remove a baseline response status together with the breaking OpenAPI change. The subset assertion then passes. No workflow or test prevents modification of scripts/openapi_v1_compatibility_baseline.json. Add a CI guard that rejects changes to the v1 baseline after its introduction, or compare it against a protected trusted revision. This is required for the gate to fail closed.

You are interacting with an AI system.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant