Repository navigation
Monitoring: Add per API key rate monitoring and Discord alerts #797
Description
Is your feature request related to a problem?
There is no monitoring or alerting for per API key request rates, leading to potential abuse on high-cost endpoints. This lack of visibility can result in unexpected costs and runaway clients.
Describe the solution you'd like
Implement monitoring and Discord alerts for per API key request rates with the following thresholds in a one-minute window:
- llm/call/ → more than 15 requests per minute
- evaluations/ → more than 3 requests per minute
- collections/ → more than 3 requests per minute
Notifications should be sent to a designated Discord channel detailing the triggering API key, endpoint, and request count. This proposal is for monitoring and alerting only; no rate limiting is required.
Metadata
Metadata
Assignees
Labels
Type
Projects
- Status
Activity
[-]Add per API key rate monitoring and Discord alerts for high-cost endpoints[/-][+]Monitoring: Alerts for API usage[/+][-]Monitoring: Alerts for API usage[/-][+]Monitoring: Add per API key rate monitoring and Discord alerts [/+]AkhileshNegi commentedon May 26, 2026
We explored using Sentry for this. Sentry lets us set a threshold based on minimum frequency over a 5-minute window and trigger alerts based on the count.
The issue is that these alerts are platform-wide, not project-specific. Since we need alerts at the project level, using only the Sentry UI won’t be enough. We’ll need to explore handling this through code and trigger Sentry alerts when the threshold is reached.