Skip to content

Fix picked-document orchestration planning and debug diagnostics - #1895

Merged
Paul Lizer (paullizer) merged 2 commits into
paullizer-react-v2-uifrom
paullizer-orchestrated-image-failure
Nov 6, 2026
Merged

Paul Lizer (paullizer) merged 2 commits into
paullizer-react-v2-uifrom
paullizer-orchestrated-image-failure

Conversation

@paullizer

@paullizer Paul Lizer (paullizer) commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Picked-document requests could fail during planning even after successful upload/source preparation. The failures occurred before execution, so the recent retained-result and narrative-delivery fixes could not help.

  • Align the advertised Analyze option schema with strict version/mode semantics, preserve legacy/editor compatibility and server-owned narrative defaults, and retain specific application-owned rejection rules.
  • Keep selected-document/operation validation inside the existing correction loop. Invalid options, omitted selections and unrenderable clarifications share one correction budget and the same planner/model; failures never silently drop files or become ungrounded answers.
  • Add debug-gated, content-free proposal/source-binding snapshots to console and Application Insights. Hashes, counts, option-shape flags and exact rejection rules distinguish model omissions from normalization changes, with explicit detail bounds and no raw prompts, document contents, IDs, filenames, settings or provider replies.
  • Preserve Analyze, upload-time OCR/vision, useful pre-plan spreadsheet inspection and native Excel evidence. Add real HTTP/planner/logger regressions across PNG, PDF, DOCX, CSV and XLSX, update the related documentation, and bump the application to 0.261.413 after integrating the latest base.

No new capability toggle, dependency, route, cloud resource or migration is introduced by this PR. No Azure deployment or live conversation replay was performed. The original deployed proposal was not retained, so the offline reproductions establish the corrected contracts and new diagnostic visibility, not a live model success-rate claim.

Linked issue

N/A. The user explicitly requested no issue creation. Related prior reliability fixes: #1874 and #1875; their useful observation-retention and available-information behavior is preserved.

Release Notes & Latest Features

  • New Feature
  • Bug Fix
  • UI Enhancement
  • Breaking Change
  • Internal only

Is this visible to end users?

  • Yes
  • No

Is this admin-facing (Admin Settings, governance, deployment, config)?

  • Yes
  • No

Should this become a Latest Feature card?

  • Yes
  • No
  • Already added

Screenshot needed for the card?

  • Yes
  • No
  • Attached

Administrators receive additional diagnostics behind the existing debug logging switch; no new setting or frontend asset is added. Release notes were offered, but the user was unavailable to approve that optional update, so they remain unchanged.

Version bump

  • application/single_app/config.py VERSION third segment bumped, or not needed because this is docs-only
  • deployers/version.txt bumped, or not needed because deployers/ was not changed

Application: 0.261.412 -> 0.261.413 relative to the updated base. The conflict with the original 0.261.411 patch was resolved after merging the guided-question base at bc3d2584e. No deployer files changed.

Testing / validation

The functional commands used this PowerShell environment:

$env:PYTHONDONTWRITEBYTECODE = '1'
$root = (Get-Location).Path
$env:PYTHONPATH = "$root\application\single_app;$root\functional_tests"
# Passed after the base merge: 79 tests covering document recovery and guided-input behavior together.
python -m pytest -q -p no:cacheprovider --disable-warnings --tb=short --show-capture=no .\functional_tests\test_orchestration_document_planning_recovery.py .\functional_tests\test_guided_user_input.py .\functional_tests\test_guided_user_input_orchestration.py

# Passed before the base merge: 214 tests, including preserved previews/Analyze availability,
# both approval modes, Auto routing, source retention, shared retry limits and debug privacy/gating.
python -m pytest -q -p no:cacheprovider --disable-warnings --tb=short --show-capture=no .\functional_tests\test_orchestration_document_planning_recovery.py .\functional_tests\test_document_analysis_observation_identity.py .\functional_tests\test_orchestration_internal_analysis.py .\functional_tests\test_orchestration_selected_source_planning.py .\functional_tests\test_orchestration_available_information.py

# Passed: 1 test; 10 deselected. Bound model and editor context survive clarification recovery.
python -m pytest -q -p no:cacheprovider --disable-warnings --tb=short --show-capture=no .\functional_tests\test_orchestration_plan_revision_planner.py -k unrenderable

# Passed: 7 tests. Existing native Excel comments and saved-cell evidence remain intact.
python -m pytest -q -p no:cacheprovider --disable-warnings --tb=short --show-capture=no .\functional_tests\test_excel_native_inspection_chat_evidence.py

# Passed: 1 test.
python -m pytest -q -p no:cacheprovider --disable-warnings --tb=short --show-capture=no .\functional_tests\test_appinsights_logging_recursion_guard.py

# Passed: 5/5 checks.
python .\functional_tests\test_privacy_logging_telemetry_audit.py

# Passed after the base merge: 7/7 checks; generated surface inventory remains current.
python .\functional_tests\test_docs_app_surface_coverage.py

# Passed after the base merge: 6/6 checks.
python .\functional_tests\test_docs_site_quality.py

# Passed before commit.
git -c core.whitespace=blank-at-eol,blank-at-eof,space-before-tab,cr-at-eol diff --check

# Passed for the complete staged change before both commits.
git diff --cached --check

Known baseline failures, not reported as green:

# 84 passed, 3 failed. The three failures reproduce with the unchanged baseline serializer.
python -m pytest -q -p no:cacheprovider --disable-warnings --tb=short --show-capture=no .\functional_tests\test_ask_ai_debug_diagnostics.py .\functional_tests\test_appinsights_logging_recursion_guard.py

The failures are the existing bind_conversation_reference, set_task_conversation_references and unbind_conversation_reference diagnostic-operation assertions. All three reproduce with the serializer from baseline 268bfa8e6 loaded in memory. This PR does not alter or weaken that operation allowlist.

The older AST-extracted planner-failure fixture already had ten setup errors from undefined STORAGE_FAILURE_CODE before implementation. New coverage uses real modules/routes and an active offline network guard instead of relying on that fixture. Dependency deprecation warnings remain. No skips or network-guard suppression were added to hide failures.

Live beta verification is not performed; rollout and an instrumented replay of the original request remain separate authorized actions.

Documentation

  • Release notes updated, or not needed
  • Feature documentation updated, or not needed
  • Fix documentation updated, or not needed

Updated CHAT_ORCHESTRATION.md, admin troubleshooting and the logging reference. Added ORCHESTRATION_DOCUMENT_PLANNING_RECOVERY_FIX.md with the cause, compatibility boundaries, debug properties, executed checks and baseline limitations. Release notes remain pending optional approval as described above.

Security checklist

  • New Flask routes include @swagger_route(security=get_auth_security())
  • Settings sent to non-admin frontends use sanitize_settings_for_user()
  • Browser JavaScript is served from local SimpleChat static assets only; no CDN-hosted JS
  • No secrets, keys, connection strings, or local-only artifacts are included

No routes, frontend settings responses or browser assets changed relative to the PR base. Source authorization and source-kind checks remain strict; diagnostic hash properties accept only validated SHA-256 values. Private incident payloads and session artifacts are not committed.

Align Analyze option contracts, retain selected sources through bounded plan correction, and add privacy-safe debug diagnostics while preserving Analyze and pre-plan inspection. Update documentation and application version to 0.261.411.

Co-authored-by: Copilot App <[email protected]>
Resolve the application version overlap at 0.261.413 and align recovery test/documentation headers. Preserve guided-question support and verify the combined document-planning and guided-input behavior.

Co-authored-by: Copilot App <[email protected]>
@paullizer
Paul Lizer (paullizer) merged commit a7c2954 into paullizer-react-v2-ui Nov 6, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant