You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Objective:
The overall objective is to help non profit users to get more actionable, data driven insights from Eval runs that can help them to further optimise the prompt. Few initiatives listed below
Initiative 1: Enable users to analyze eval performance across thematic categories (e.g PII handling, Hinglish handling, corner case, hallucination checks, Facts handling etc) to identify weak areas and improve prompts/models faster accordingly. This was specific request from Sneha, VG etc. Github ticket
Initiative 2 (long term): Use AI model to recommend prompt changes. One of the input for this can be category wise scoring as part of initiative 1
Activity
[-]Evals Insights layer[/-][+]Evaluation: Thematic category analysis[/+][-]Evaluation: Thematic category analysis[/-][+]Evaluation: Insights layer[/+]