Skip to content

Improve orchestration UX and semantic answer checking - #1899

Merged
Paul Lizer (paullizer) merged 1 commit into
microsoft:paullizer-react-v2-uifrom
paullizer:paullizer-chat-orchestration-experience
Nov 6, 2026
Merged

Paul Lizer (paullizer) merged 1 commit into
microsoft:paullizer-react-v2-uifrom
paullizer:paullizer-chat-orchestration-experience

Conversation

@paullizer

Copy link
Copy Markdown
Contributor

Summary

  • Replace the V2 execution-dashboard feel with Quiet activity, composer-only Stop, concise recovery, and clarification forms that collapse after answering. Resolve authorized conversation attachments before asking unnecessary source questions.
  • Fix the investigated completeness_requirements_changed failures: preserve canonical obligation identities without requiring verbatim checklist descriptions. Show preparation, deterministic validation, and semantic review as separate stages; expose Auto / On / Off for optional semantic review through approved plan edits.
  • Give repairable failures feedback-informed corrections within the existing shared budget. Required schema, access, provenance, and safety checks remain mandatory. Eligible Retry check reuses retained content and diagnoses; a missing checkpoint refuses recovery instead of silently regenerating. An unavailable elected check never publishes an unchecked answer.
  • Hand actual authorized image bytes to existing vision support when saved text is absent. Route workbook Notes and threaded comments through native inspection, preserving available metadata, cell values, coverage, and citation lineage. Keep classic workflow outcomes compatible with both historical and new report versions.

Linked issue

N/A - no linked issue.

Release Notes & Latest Features

  • New Feature
  • Bug Fix
  • UI Enhancement
  • Breaking Change
  • Internal only

Is this visible to end users?

  • Yes
  • No

Is this admin-facing (Admin Settings, governance, deployment, config)?

  • Yes
  • No

Should this become a Latest Feature card?

  • Yes
  • No
  • Already added

Screenshot needed for the card?

  • Yes
  • No
  • Attached

Release notes remain unchanged pending approval. Existing administrative correction limits and semantic-review disablement remain outer bounds; plan choices cannot override them.

Version bump

  • application/single_app/config.py VERSION third segment bumped, or not needed because this is docs-only
  • deployers/version.txt bumped, or not needed because deployers/ was not changed

Application version: 0.261.415 -> 0.261.416. No deployer changes.

Testing / validation

  • python .\scripts\check_cosmos_response_boundaries.py --run-contracts - passed: 2,005 tests and 431 subtests with Cosmos network access blocked; 1,094 owners, 425 doubles, zero review gaps.
  • python -m pytest .\ui_tests\test_v2_orchestration_recovery.py .\ui_tests\test_v2_orchestration_quiet_activity.py .\ui_tests\test_v2_orchestration_m365_inline_approval.py .\ui_tests\test_v2_orchestration_m365_recovery.py -q -x - passed: 84 tests.
  • python -m pytest .\ui_tests\test_task_completeness_ui.py -q - passed: 13 tests, including classic report versions 1/2, historical inspection, keyboard accessibility, and inert markup.
  • python -m pytest functional_tests\test_orchestration_attachment_native_evidence.py functional_tests\test_orchestration_selected_source_planning.py functional_tests\test_orchestration_search_result_integrity.py functional_tests\test_orchestration_source_kind_planning.py functional_tests\test_chat_vision_current_turn.py -q --disable-warnings --basetemp=.test-artifacts\orchestration-native --tb=short - passed: 115 tests.
  • python .\functional_tests\test_docs_app_surface_coverage.py - passed: 7/7 checks.
  • python .\functional_tests\test_docs_site_quality.py - passed: 6/6 checks.
  • git --no-pager diff --check - passed before committing.

Quiet activity was inspected at 1440px/light and 390px/dark without overflow. Scripted provider responses verify control flow and evidence handoff, not an empirical semantic-accuracy score. Native workbook coverage remains limited to authorized readable material and is explicitly partial when resource limits apply.

Known unrelated limitation: python .\ui_tests\test_v2_orchestration_composer.py fails an older public-source tooltip expectation at line 291; the committed base already uses different wording, and the legacy harness lacks an existing document-sources endpoint stub. Left unchanged. No production deployment or replay of the investigated conversations was performed.

Documentation

  • Release notes updated, or not needed
  • Feature documentation updated, or not needed
  • Fix documentation updated, or not needed

Added docs/explanation/fixes/ORCHESTRATION_QUIET_ACTIVITY_AND_SEMANTIC_REVIEW_FIX.md; updated orchestration administration, plan-review guidance, and chat-control reference. Documentation inventory remains current.

Security checklist

  • New Flask routes include @swagger_route(security=get_auth_security())
  • Settings sent to non-admin frontends use sanitize_settings_for_user()
  • Browser JavaScript is served from local SimpleChat static assets only; no CDN-hosted JS
  • No secrets, keys, connection strings, or local-only artifacts are included

No new Flask routes or raw settings responses were introduced. Public checking reports omit candidates, prompts, and raw evidence. Attachment access, masking, linkage, and revision are rechecked around reads; existing approval and external-effect recovery fencing remains intact.

Keep end-user progress quiet, collapse answered questions, separate preparation from checking, and apply feedback-informed corrections. Preserve authorized image and native workbook evidence with safe retained-check recovery and regression coverage.

Co-authored-by: Copilot App <[email protected]>
@paullizer
Paul Lizer (paullizer) merged commit 0452e9c into microsoft:paullizer-react-v2-ui Nov 6, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant