Skip to content

fix(llm): keep async agents responsive during model requests - #987

Merged
alcholiclg merged 1 commit into
modelscope:mainfrom
alcholiclg:feat/fix-llm-event-loop-blocking
Sep 18, 2026
Merged

alcholiclg merged 1 commit into
modelscope:mainfrom
alcholiclg:feat/fix-llm-event-loop-blocking

Conversation

@alcholiclg

Copy link
Copy Markdown
Collaborator

Change Summary

  • Move synchronous model requests and stream reads into a dedicated executor to prevent slow model I/O from blocking the WebUI event loop and health probes. Fix cancellation cleanup for OpenAI-compatible and Anthropic providers across router and legacy SDK paths.

Related issue number

Checklist

  • The pull request title is a good summary of the changes - it will be used in the changelog
  • Unit tests for the changes exist
  • Run pre-commit install and pre-commit run --all-files before git commit, and passed lint check.
  • Documentation reflects the changes where applicable

@alcholiclg
alcholiclg merged commit 0594722 into modelscope:main Sep 18, 2026
1 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants