Repository navigation
Evaluation: Tracking overall latency #1041
Description
Is your feature request related to a problem?
We lack visibility on the overall evaluation process latency, which affects our ability to identify bottlenecks and optimize performance.
Describe the solution you'd like
- Track overall evaluation latency
- Breakdown latency for cosine score and LLM judge score
- Identify latency added by kaapi
- Explore scope for improvement in latency
Original issue
Tracking below for fast evals (observability)
- Tracking Evals overall latency
- Breakup for latency for cosine score, latency for llm judge score
- Latency added by kaapi ?
Basis above, explore scope for improvement on the latency
Metadata
Metadata
Assignees
Labels
No labels
Type
Projects
- Status
Activity
[-]Visibility on overall Eval process latency[/-][+]Evaluation: Tracking overall latency[/+]