You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem?
Kaapi enables Langfuse tracing by default when Langfuse credentials are added. While Langfuse is also required to run evaluations, it can lead to high-volume orgs quickly exhausting their Langfuse rate-limit/credit budget on traces alone. This results in 429 errors during evaluation resync, especially on the free tier. So there is no option of using solely evaluations without enabling tracing
Describe the solution you'd like
Add a flag to enable/disable Langfuse tracing per project, making tracing opt-in.
Tracing should be off by default (or explicitly toggleable) for a project
Orgs needing only evaluations can keep tracing disabled to save their rate-limit/credit budget
Orgs wanting full observability can enable it when necessary
Activity
[-]Langfuse tracing[/-][+]Kaapi: Opt-in Langfuse tracing[/+]