Repository navigation
guardrails: Fix rephrase_needed flag #113
Description
Is your feature request related to a problem?
The API incorrectly returns rephrase_needed: false for validators like topic_relevance when they are configured with on_fail: rephrase. This causes confusion as safe_text shows a rephrase message but the flag indicates no rephrasing is necessary.
Describe the solution you'd like
- Update the condition for
rephrase_neededto return true ifsafe_textstarts with the rephrase prefix ("Please rephrase the query without unsafe content."). - Ensure this applies to all validators that implement the rephrase on-fail action, not just the LLM Critic validator.
Original issue
Describe the bug
When a validator (e.g. topic_relevance) is configured with on_fail: rephrase and the input fails validation, the API response correctly populates safe_text with the rephrase message but incorrectly returns rephrase_needed: false.
Steps to reproduce
Create a topic relevance config with a restricted scope.
Send a guardrail request with on_fail: rephrase and an input outside the allowed topic scope.
Observe the response — safe_text is populated but rephrase_needed is false.
Expected behavior
rephrase_needed should be true whenever safe_text begins with the rephrase prefix ("Please rephrase the query without unsafe content.").
Actual behavior
rephrase_needed is false. The field was only set to true when the output exactly matched LLM_CRITIC_REPHRASE_MESSAGE, which is specific to the LLM Critic validator.
Fix
Extended the rephrase_needed condition to also flag outputs starting with REPHRASE_ON_FAIL_PREFIX, covering all validators that use the rephrase on-fail action.
Metadata
Metadata
Assignees
Labels
Type
Projects
- Status
Activity
[-]rephrase_needed always returns false for non-LLM-Critic validators with on_fail: rephrase[/-][+]Evaluation: Fix rephrase_needed flag[/+][-]Evaluation: Fix rephrase_needed flag[/-][+]guardrails: Fix rephrase_needed flag[/+]