Evaluation: Increase max unique rows #995

Is your feature request related to a problem?
Fast evaluation is currently limited to small datasets, which restricts the ability to effectively utilize direct LLM calls for evaluations, causing inefficiencies.

Describe the solution you'd like

• Increase EVAL_FAST_MAX_UNIQUE_ROWS from 10 to 100
• Allow up to ~500 total items per fast run with a default duplication factor of 5
• Maintain eligibility checks based on unique rows
• Monitor API concurrency and memory usage during testing
• Confirm failure threshold is appropriate at the new scale
• Follow rollout plan to test and deploy the changes.