Performance issues under load testing (using teams_sso sample)

Hello,

I am currently migrating our existing solution from the Microsoft Bot Framework to the new Agent 365 SDK for python. As a reference, I am using the teams_sso sample from this repository.

After completing the migration, we successfully passed all functional tests and were preparing for production deployment. However, we encountered significant performance issues during load testing.

Test Setup:

I deployed the teams_sso sample locally and connected it to an Azure Bot Service resource.
For load testing, I used the BotServiceStressToolkit. (https://github.com/damadei/BotServiceStressToolkit)
The test involved 30 concurrent users.
Each user triggers two types of requests per interaction: ConversationUpdate and Message.
I am not using the SSO functionality for these tests—just simple requests that should immediately respond with:
"Please type 'login' to sign in, 'logout' to sign out, or 'getUserProfile' to get user info."

Results:

The average response time for ConversationUpdate requests is 23 seconds.
The average response time for Message requests is 11 seconds.
These response times are far too high for production use.

Questions:

What could be causing such long processing times when using the Agent 365 SDK, even with the basic sample and without SSO?
Are there known limitations or configuration steps required to improve performance under load?
Are there best practices or recommendations for optimizing the Agent 365 SDK for high concurrency scenarios?
Is there any guidance on how to properly scale or configure the SDK and/or Azure Bot Service for better throughput?

Any insights or suggestions would be greatly appreciated, as we would like to proceed with our migration but need to ensure the system is production-ready.

Thank you very much for your support!
Best regards,
Mario

Thank you, Mario. I will investigate this and get back to you within a day or two.

Also, which version of the SDK are you running?

Rodrigo Brandão (@rodrigobr-msft)

Also, which version of the SDK are you running?

Hi Rodrigo Brandão (@rodrigobr-msft). My app is running on agent sdk version 0.3.2

Apologies, but this will take a bit longer to resolve than anticipated.

Thank you for the honest update — I really appreciate it. No worries on our side, as we fortunately caught the issue before going live, so it’s not time-critical for us.

Do you know how the performance of Agents 365 SDK compares to Bot Framework? Is Bot Framework significantly faster?

Same test setup. Only with the bot-conversation-quickstart sample and with a simple modification to get a quick response.
"Processing your message..."

https://github.com/OfficeDev/Microsoft-Teams-Samples/tree/main/samples/bot-conversation-quickstart/python

class BotActivityHandler(TeamsActivityHandler):
    def __init__(self):
        super().__init__()

        # Registers an activity event handler for the message activity event.
        # This is triggered whenever the bot receives a message.
        self.on_message_activity = self.handle_message

    async def handle_message(self, turn_context: TurnContext):
        """
        Handles incoming messages sent to the bot.

        Args:
            turn_context (TurnContext): Provides context for the current turn of conversation.
        """
        await turn_context.send_activity("Processing your message...")
        return

Results:

The average response time for ConversationUpdate requests is about 3 seconds.
The average response time for Message requests is just under 4 seconds.

Hi, I wanted to post a quick update:

#175

We are observing greater performance with these changes, but we are still on the lookout for more opportunities for improvement. I'll send you an update when these changes make it into a release.

maaisde

maaisde

These are preliminary results when running our own benchmarking tool on the Quickstart sample. The first set of results is with the optimizations and the second set is without. Of course, this is not the same sample that your team ran, but I believe these optimizations will affect that sample similarly, especially since you ran yours without actually triggering any auth flows. This is with spawning 30 coroutines, each one posting a message activity to the server.

(venv) > python -m src.main -n 30 -a

---- Aggregated Results ----

Average Time: 0.3129 seconds
Min Time: 0.1359 seconds
Max Time: 0.4441 seconds

Success Rate: 30 / 30

Total Time: 0.5305111408233643 seconds

(venv) > python -m src.main -n 30 -a

---- Aggregated Results ----

Average Time: 45.2215 seconds
Min Time: 7.0094 seconds
Max Time: 70.4800 seconds

Success Rate: 30 / 30

Total Time: 70.55655002593994 seconds

Rodrigo Brandão (@rodrigobr-msft)

Rodrigo Brandão (@rodrigobr-msft)
Thank you very much for your support. I can confirm that with the JWT/MSAL optimizations it is now fast.

Navigation Menu

Repository navigation

Performance issues under load testing (using teams_sso sample) #133

Total Time: 0.5305111408233643 seconds

Total Time: 70.55655002593994 seconds

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Performance issues under load testing (using teams_sso sample) #133

Description

Activity

rodrigobr-msft commented on Sep 22, 2025

rodrigobr-msft commented on Sep 22, 2025

maaisde commented on Sep 22, 2025

rodrigobr-msft commented on Sep 25, 2025

maaisde commented on Sep 29, 2025

rodrigobr-msft commented on Oct 1, 2025

maaisde commented on Oct 2, 2025

rodrigobr-msft commented on Oct 16, 2025

rodrigobr-msft commented on Oct 17, 2025

Total Time: 0.5305111408233643 seconds

Total Time: 70.55655002593994 seconds

maaisde commented on Oct 21, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions