diff --git a/backend/app/api/docs/guardrails/create_ban_list.md b/backend/app/api/docs/guardrails/create_ban_list.md new file mode 100644 index 000000000..3d78b8610 --- /dev/null +++ b/backend/app/api/docs/guardrails/create_ban_list.md @@ -0,0 +1,34 @@ +Create a ban list — a named set of words the `ban_list` validator redacts from text. + +A ban list is stored by the guardrails service and scoped to the calling project. Reference it from a validator config by setting that config's `ban_list_id` to the `id` returned here. + +### Request + +```json +{ + "name": "Safety Banned Terms", + "description": "Terms not allowed for this tenant policy", + "domain": "abuse", + "is_public": false, + "banned_words": ["slur_a", "slur_b"] +} +``` + +| Field | Type | Required | Notes | +|---|---|---|---| +| `name` | string | yes | 1–100 chars. Must be unique for the project. | +| `description` | string | yes | 1–500 chars. | +| `banned_words` | string[] | yes | Up to 1000 entries, each 1–100 chars. | +| `domain` | string | yes | Free-form grouping label used to filter on list. | +| `is_public` | boolean | no | Defaults to `false`. | + +### Notes + +- `is_public: true` makes the list readable by other tenants. Updating and deleting stay restricted to the owning project regardless. +- The response is `200`, not `201` — the guardrails service does not use `201`. + +### Errors + +- `400` — a ban list with this configuration already exists. +- `422` — the guardrails service rejected the body. +- `502` — the guardrails service is unreachable or returned a non-JSON body. diff --git a/backend/app/api/docs/guardrails/create_llm_prompt_config.md b/backend/app/api/docs/guardrails/create_llm_prompt_config.md new file mode 100644 index 000000000..4ebd1103d --- /dev/null +++ b/backend/app/api/docs/guardrails/create_llm_prompt_config.md @@ -0,0 +1,49 @@ +Create a stored prompt for one of the LLM-backed validators. + +Two validators read their instructions from a stored prompt instead of hard-coding them: `topic_relevance` (does this text stay on topic?) and `answer_relevance_custom_llm` (does this answer address the question?). Create the prompt here, then reference its `id` from the matching validator config. + +### Request + +```json +{ + "validator_name": "topic_relevance", + "name": "Maternal Health Scope", + "description": "Topic guard for maternal health support bot", + "prompt_schema_version": 1, + "llm_prompt": "Pregnancy care: Questions about prenatal care, ANC visits, nutrition, supplements, danger signs. Postpartum care: Questions about recovery after delivery, breastfeeding, and mother health checks." +} +``` + +| Field | Type | Required | Notes | +|---|---|---|---| +| `validator_name` | enum | yes | `topic_relevance` or `answer_relevance_custom_llm`. Immutable after creation. | +| `name` | string | yes | 1–100 chars. | +| `description` | string | yes | 1–500 chars. | +| `prompt_schema_version` | integer | no | Defaults to `1`. Must be >= 1. | +| `llm_prompt` | string | yes | The prompt text. Non-empty. | + +### Placeholders + +For `answer_relevance_custom_llm` the prompt **must** contain both `{query}` and `{answer}`; the service rejects it otherwise. Example: + +``` +You are evaluating a maternal health assistant. +Query: {query} +Answer: {answer} + +Does the answer directly address the maternal health query? +Answer only YES or NO. +``` + +`topic_relevance` prompts have no required placeholders. + +### Notes + +- New configs are created active. `is_active` can only be changed via `PATCH`. +- Responds `200`, not `201`. + +### Errors + +- `400` — a config with the same validator, version and prompt text already exists. +- `422` — the guardrails service rejected the body, e.g. an `answer_relevance_custom_llm` prompt missing `{query}`/`{answer}`. +- `502` — the guardrails service is unreachable or returned a non-JSON body. diff --git a/backend/app/api/docs/guardrails/create_validator_config.md b/backend/app/api/docs/guardrails/create_validator_config.md new file mode 100644 index 000000000..160a4e9b8 --- /dev/null +++ b/backend/app/api/docs/guardrails/create_validator_config.md @@ -0,0 +1,44 @@ +Register a validator configuration — a named, reusable validator setup that `POST /guardrails` applies by id. + +This is the object you point at from a guardrails run: create one here, then pass its `id` as a `validator_config_id` in the `config` array of `POST /guardrails`. + +### Request + +The body is a fixed set of base fields **plus** whatever tuning keys the chosen validator type accepts. Call `GET /guardrails` to discover the accepted keys per type. + +```json +{ + "name": "PII Redaction Input", + "type": "pii_remover", + "stage": "input", + "on_fail_action": "fix", + "is_enabled": true, + "entity_types": ["EMAIL_ADDRESS", "PHONE_NUMBER", "IN_AADHAAR"] +} +``` + +| Field | Type | Required | Notes | +|---|---|---|---| +| `name` | string | yes | 5–225 chars. | +| `type` | enum | yes | One of the validator types from `GET /guardrails`. | +| `stage` | enum | yes | `input` or `output`. | +| `on_fail_action` | enum | no | `exception` \| `fix` \| `rephrase`. Defaults to `fix`. | +| `is_enabled` | boolean | no | Defaults to `true`. | +| *(extra keys)* | any | no | Validator-specific tuning, stored as the config blob. | + +In the example above, `entity_types` is a `pii_remover` tuning key — not part of the base schema, which is why it appears as an additional property rather than a named field. + +This endpoint forwards the body to the guardrails service unchanged; that service validates it. A tuning key or validator type newer than this documentation still works — check `GET /guardrails` for what your deployment accepts. + +### Notes + +- **Uniqueness is enforced on `name` alone**, scoped to the project. The same validator type may be registered many times under different names. +- `stage` is advisory. `POST /guardrails` routes on the text it is actually given, not on this field, so one config can serve both directions. +- Do **not** send `organization_id` or `project_id`. The tenant is derived from your authenticated context, and the guardrails service rejects those keys in the body. +- Responds `200`, not `201`. + +### Errors + +- `400` — a validator config with this name already exists in the project. +- `422` — the guardrails service rejected the body, or it contained `organization_id`/`project_id`. +- `502` — the guardrails service is unreachable or returned a non-JSON body. diff --git a/backend/app/api/docs/guardrails/delete_ban_list.md b/backend/app/api/docs/guardrails/delete_ban_list.md new file mode 100644 index 000000000..923028749 --- /dev/null +++ b/backend/app/api/docs/guardrails/delete_ban_list.md @@ -0,0 +1,12 @@ +Delete a ban list permanently. Restricted to the owning project. + +Validator configs that still reference the deleted list by `ban_list_id` are not cleaned up; they will fail when the guardrails service next tries to resolve the list. + +Responds `200` with a confirmation body rather than `204`. + +### Errors + +- `403` — the list belongs to another tenant. +- `404` — no such ban list. +- `422` — `ban_list_id` is not a valid UUID. +- `502` — the guardrails service is unreachable or returned a non-JSON body. diff --git a/backend/app/api/docs/guardrails/delete_llm_prompt_config.md b/backend/app/api/docs/guardrails/delete_llm_prompt_config.md new file mode 100644 index 000000000..ef8d61eff --- /dev/null +++ b/backend/app/api/docs/guardrails/delete_llm_prompt_config.md @@ -0,0 +1,11 @@ +Delete a stored LLM prompt config permanently, scoped to the calling project. + +Validator configs that still reference the deleted prompt will fail when the guardrails service next tries to resolve it. Repoint or remove them first. + +Responds `200` with a confirmation body rather than `204`. + +### Errors + +- `404` — no such config, or it belongs to another project. +- `422` — `prompt_config_id` is not a valid UUID. +- `502` — the guardrails service is unreachable or returned a non-JSON body. diff --git a/backend/app/api/docs/guardrails/delete_validator_config.md b/backend/app/api/docs/guardrails/delete_validator_config.md new file mode 100644 index 000000000..9d8db14bc --- /dev/null +++ b/backend/app/api/docs/guardrails/delete_validator_config.md @@ -0,0 +1,11 @@ +Delete a validator config permanently, scoped to the calling project. + +Guardrails runs that still pass the deleted `validator_config_id` will no longer resolve it. Remove the id from your `POST /guardrails` calls first. + +Responds `200` with a confirmation body rather than `204`. + +### Errors + +- `404` — no such config, or it belongs to another project. +- `422` — `config_id` is not a valid UUID. +- `502` — the guardrails service is unreachable or returned a non-JSON body. diff --git a/backend/app/api/docs/guardrails/get_ban_list.md b/backend/app/api/docs/guardrails/get_ban_list.md new file mode 100644 index 000000000..954c1e9f9 --- /dev/null +++ b/backend/app/api/docs/guardrails/get_ban_list.md @@ -0,0 +1,10 @@ +Fetch a single ban list by id. + +Readable if the list belongs to the calling project, or if it belongs to another tenant and is marked `is_public`. + +### Errors + +- `403` — the list belongs to another tenant and is not public. +- `404` — no such ban list. +- `422` — `ban_list_id` is not a valid UUID. +- `502` — the guardrails service is unreachable or returned a non-JSON body. diff --git a/backend/app/api/docs/guardrails/get_guardrails_job.md b/backend/app/api/docs/guardrails/get_guardrails_job.md new file mode 100644 index 000000000..f74830603 --- /dev/null +++ b/backend/app/api/docs/guardrails/get_guardrails_job.md @@ -0,0 +1,28 @@ +Poll a guardrails job for its status and sanitised result. + +Use this when you submitted `POST /guardrails` without a `callback_url`, or to inspect a completed job for traceability. The job id comes from the `POST /guardrails` response. + +### Status values + +| `status` | Meaning | +|---|---| +| `PENDING` | Queued, not yet picked up. | +| `PROCESSING` | A worker is applying the validators. | +| `SUCCESS` | Finished; `guardrails_response` carries the sanitised text. | +| `FAILED` | The text was hard-blocked, or the job errored. `error_message` explains why. | + +### Response + +`guardrails_response` is populated only on `SUCCESS`; it is `null` in every other state. The sanitised text sits at `guardrails_response.response.output.content.value`. + +`warnings` mirrors the `metadata.warnings` of the webhook payload, so polling callers do not miss a bypass signal — most importantly the case where the guardrails service was unavailable and the original text was returned unchanged. It is always empty for a hard-blocked job. + +### Notes + +- If the upstream response carried no sanitised text, the value falls back to the original submitted text. +- `usage` counters default to zero when the guardrails service reports none. + +### Errors + +- `404` — no such job in this project, or the id belongs to a job that is not a guardrails job. +- `422` — `job_id` is not a valid UUID. diff --git a/backend/app/api/docs/guardrails/get_llm_prompt_config.md b/backend/app/api/docs/guardrails/get_llm_prompt_config.md new file mode 100644 index 000000000..0e583f368 --- /dev/null +++ b/backend/app/api/docs/guardrails/get_llm_prompt_config.md @@ -0,0 +1,7 @@ +Fetch a single LLM prompt config by id, scoped to the calling project. + +### Errors + +- `404` — no such config, or it belongs to another project. +- `422` — `prompt_config_id` is not a valid UUID. +- `502` — the guardrails service is unreachable or returned a non-JSON body. diff --git a/backend/app/api/docs/guardrails/get_validator_config.md b/backend/app/api/docs/guardrails/get_validator_config.md new file mode 100644 index 000000000..f49a81f10 --- /dev/null +++ b/backend/app/api/docs/guardrails/get_validator_config.md @@ -0,0 +1,9 @@ +Fetch a single validator config by id, scoped to the calling project. + +The response is the stored row flattened together with its validator-specific tuning config, so it carries keys beyond the declared schema — `id`, `organization_id`, `project_id`, `created_at`, `updated_at`, and every tuning key for that validator type. + +### Errors + +- `404` — no such config, or it belongs to another project. +- `422` — `config_id` is not a valid UUID. +- `502` — the guardrails service is unreachable or returned a non-JSON body. diff --git a/backend/app/api/docs/guardrails/list_ban_lists.md b/backend/app/api/docs/guardrails/list_ban_lists.md new file mode 100644 index 000000000..938fa2390 --- /dev/null +++ b/backend/app/api/docs/guardrails/list_ban_lists.md @@ -0,0 +1,16 @@ +List the ban lists visible to the calling project. + +Returns lists owned by the project plus any list from another tenant marked `is_public`. Ordered newest first. + +### Query parameters + +| Parameter | Type | Required | Default | Description | +|---|---|---|---|---| +| `domain` | string | no | — | Return only lists carrying this domain label. | +| `offset` | integer | no | `0` | Rows to skip. Must be >= 0. | +| `limit` | integer | no | — | Max rows to return, 1–100. Omit for no limit. | + +### Errors + +- `422` — `offset` is negative, or `limit` is outside 1–100. +- `502` — the guardrails service is unreachable or returned a non-JSON body. diff --git a/backend/app/api/docs/guardrails/list_llm_prompt_configs.md b/backend/app/api/docs/guardrails/list_llm_prompt_configs.md new file mode 100644 index 000000000..f8d03d376 --- /dev/null +++ b/backend/app/api/docs/guardrails/list_llm_prompt_configs.md @@ -0,0 +1,14 @@ +List the stored LLM prompt configs belonging to the calling project, oldest first. + +### Query parameters + +| Parameter | Type | Required | Default | Description | +|---|---|---|---|---| +| `validator_name` | enum | no | — | `topic_relevance` or `answer_relevance_custom_llm`. | +| `offset` | integer | no | `0` | Rows to skip. Must be >= 0. | +| `limit` | integer | no | — | Max rows to return, 1–100. Omit for no limit. | + +### Errors + +- `422` — `validator_name` is not a recognised value, `offset` is negative, or `limit` is outside 1–100. +- `502` — the guardrails service is unreachable or returned a non-JSON body. diff --git a/backend/app/api/docs/guardrails/list_validator_configs.md b/backend/app/api/docs/guardrails/list_validator_configs.md new file mode 100644 index 000000000..17cf6c336 --- /dev/null +++ b/backend/app/api/docs/guardrails/list_validator_configs.md @@ -0,0 +1,20 @@ +List the validator configs belonging to the calling project, oldest first. + +### Query parameters + +| Parameter | Type | Required | Default | Description | +|---|---|---|---|---| +| `ids` | UUID | no | — | Repeat the parameter to fetch several by id (`?ids=A&ids=B`). | +| `stage` | enum | no | — | `input` or `output`. | +| `type` | enum | no | — | A validator type from `GET /guardrails`. | + +This route is not paginated. + +### Response + +Each item is the stored row flattened together with its validator-specific tuning config, so entries carry keys beyond the declared schema — `id`, `organization_id`, `project_id`, `created_at`, `updated_at`, and every tuning key for that validator type. + +### Errors + +- `422` — a value in `ids` is not a valid UUID, or `stage`/`type` is not a recognised value. +- `502` — the guardrails service is unreachable or returned a non-JSON body. diff --git a/backend/app/api/docs/guardrails/list_validator_types.md b/backend/app/api/docs/guardrails/list_validator_types.md new file mode 100644 index 000000000..469aac301 --- /dev/null +++ b/backend/app/api/docs/guardrails/list_validator_types.md @@ -0,0 +1,77 @@ +Lists every validator type the guardrails service can run, with the JSON Schema of the tuning keys each one accepts. + +Start here when you are setting guardrails up. This endpoint is the authority on what **your** deployment supports: if a type is listed you can configure it, and if it is missing that deployment does not have it yet — whatever other documentation says. + +
+ +## Setting one up + +Two steps. + +**Step 1 — save a reusable config** with `POST /guardrails/validators/configs`. The body is the five shared fields, plus the tuning keys of the type you picked: + +```json +{ + "name": "PII Redaction Input", + "type": "pii_remover", + "stage": "input", + "on_fail_action": "fix", + "is_enabled": true, + "entity_types": ["EMAIL_ADDRESS", "PHONE_NUMBER", "IN_AADHAAR"] +} +``` + +| Shared field | Required | What it means | +|---|---|---| +| `name` | yes | Your label for this setup, 5–225 characters, unique in the project. The same type can be saved many times under different names. | +| `type` | yes | One of the types listed below. | +| `stage` | yes | `input` checks the user's message, `output` checks the model's reply. Advisory only — a run checks whatever text you hand it. | +| `on_fail_action` | no, defaults to `fix` | `fix` repairs the text where it can, `rephrase` returns a stand-in reply instead, `exception` fails the request outright. | +| `is_enabled` | no, defaults to `true` | Whether this config is allowed to run. | + +The response carries an `id`. That id is what you use everywhere else. + +**Step 2 — run it** with `POST /guardrails`: + +```json +{ + "text": "Call me on 9876543210", + "config": [{"validator_config_id": ""}] +} +``` + +Guardrails compose: list several ids and they run in that order over the same message. + +
+ +## One naming trap + +The schemas below describe each validator *as it runs*, so they spell the failure action `on_fail`. When you *save* a config in step 1, the same setting is called `on_fail_action`. Same three values, two names, depending on which endpoint you are talking to. + +
+ +## What this endpoint returns + +The one guardrails route that is **not** wrapped in the usual `{"success": ..., "data": ...}` envelope. The body is a bare object: + +```json +{ + "validators": [ + { + "type": "pii_remover", + "config": {"type": "object", "properties": {"entity_types": {"...": "..."}}} + } + ] +} +``` + +Each `config` is generated from the service's own model, so it is always current — and a little noisy, carrying the `title`, `$defs` and `anyOf` wrappers that JSON Schema needs for optional fields. The reference below is that same information, trimmed and explained. + +
+ +## Errors + +| Code | Meaning | +|---|---| +| `401` / `403` | Your API key is missing, invalid, or not allowed to use guardrails. | +| `502` | The guardrails service is unreachable, or returned something that was not JSON. | diff --git a/backend/app/api/docs/guardrails/update_ban_list.md b/backend/app/api/docs/guardrails/update_ban_list.md new file mode 100644 index 000000000..c70f2d10c --- /dev/null +++ b/backend/app/api/docs/guardrails/update_ban_list.md @@ -0,0 +1,22 @@ +Update a ban list. Fields you omit are left unchanged. + +`banned_words` is replaced wholesale, not merged — send the full list you want stored. + +### Request + +```json +{ + "description": "Updated description", + "banned_words": ["slur_a", "slur_b", "slur_c"] +} +``` + +All five fields (`name`, `description`, `banned_words`, `domain`, `is_public`) are optional and follow the same constraints as on create. + +### Errors + +- `400` — the update collides with an existing ban list. +- `403` — the list belongs to another tenant. Public lists are readable but not writable across tenants. +- `404` — no such ban list. +- `422` — the body failed validation, or `ban_list_id` is not a valid UUID. +- `502` — the guardrails service is unreachable or returned a non-JSON body. diff --git a/backend/app/api/docs/guardrails/update_llm_prompt_config.md b/backend/app/api/docs/guardrails/update_llm_prompt_config.md new file mode 100644 index 000000000..c28bc214a --- /dev/null +++ b/backend/app/api/docs/guardrails/update_llm_prompt_config.md @@ -0,0 +1,22 @@ +Update a stored LLM prompt config. Fields you omit are left unchanged. + +### Request + +```json +{"llm_prompt": "Pregnancy care: Updated scope definition"} +``` + +`name`, `description`, `prompt_schema_version`, `llm_prompt` and `is_active` can be patched. + +### Notes + +- `validator_name` is immutable — it cannot be patched. Create a new config instead. +- `is_active` is settable only here, not on create. +- Editing `llm_prompt` on an `answer_relevance_custom_llm` config still requires both `{query}` and `{answer}` placeholders. + +### Errors + +- `400` — the update collides with an existing config. +- `404` — no such config, or it belongs to another project. +- `422` — the body failed validation, the placeholder rule was broken, or `prompt_config_id` is not a valid UUID. +- `502` — the guardrails service is unreachable or returned a non-JSON body. diff --git a/backend/app/api/docs/guardrails/update_validator_config.md b/backend/app/api/docs/guardrails/update_validator_config.md new file mode 100644 index 000000000..9fbd34cb5 --- /dev/null +++ b/backend/app/api/docs/guardrails/update_validator_config.md @@ -0,0 +1,21 @@ +Update a validator config's base fields. Fields you omit are left unchanged. + +### Request + +```json +{"stage": "output", "is_enabled": false} +``` + +Only `name`, `type`, `stage`, `on_fail_action` and `is_enabled` can be patched. + +### Notes + +- **Validator-specific tuning cannot be changed here.** The guardrails service rejects any key outside the five base fields, so changing something like `threshold` or `entity_types` means deleting the config and recreating it. +- Because `POST /guardrails` resolves validators by id at run time, an update takes effect on the next run — there is no versioning. + +### Errors + +- `400` — the new `name` collides with an existing config in the project. +- `404` — no such config, or it belongs to another project. +- `422` — the guardrails service rejected the body, e.g. a key outside the five base fields. A malformed `config_id` is rejected here, before the call is forwarded. +- `502` — the guardrails service is unreachable or returned a non-JSON body. diff --git a/backend/app/api/docs/guardrails_schemas.py b/backend/app/api/docs/guardrails_schemas.py new file mode 100644 index 000000000..230ca6948 --- /dev/null +++ b/backend/app/api/docs/guardrails_schemas.py @@ -0,0 +1,1157 @@ +"""JSON Schemas and examples for the guardrails management-API proxy routes. + +These routes forward to the kaapi-guardrails service, which owns validation. +Describing the contract as data here (instead of as Pydantic models the +gateway would enforce) keeps Swagger fully typed and testable while letting +every payload reach upstream: a validator type or field this gateway has not +heard of yet is forwarded, not rejected. + +Keep in sync with `kaapi-guardrails/backend/app/schemas/` on a best-effort +basis — a stale description misleads, but a stale schema no longer blocks a +working call. +""" + +import json +from typing import Any, NamedTuple + +from app.api.openapi_docs import ResponseDoc, ref + +_UUID_EXAMPLE = "0b9f1a4c-2d3e-4f50-8a6b-7c8d9e0f1a2b" +_TIMESTAMP_EXAMPLE = "2026-10-14T09:30:00" + + +def _envelope(data_schema: dict[str, Any]) -> dict[str, Any]: + """The `{success, data, error, metadata}` wrapper the service returns.""" + return { + "type": "object", + "properties": { + "success": {"type": "boolean"}, + "data": data_schema, + "error": {"type": "string", "nullable": True}, + "metadata": {"type": "object", "nullable": True}, + }, + "required": ["success"], + } + + +def _list_envelope(item_schema: dict[str, Any]) -> dict[str, Any]: + return _envelope({"type": "array", "items": item_schema}) + + +_BAN_LIST_FIELDS: dict[str, Any] = { + "name": { + "type": "string", + "description": "Human-readable name for the ban list. Must be unique.", + }, + "description": { + "type": "string", + "description": "What this ban list covers and when it should be applied.", + }, + "banned_words": { + "type": "array", + "items": {"type": "string"}, + "description": ( + "Words to redact. Consumed by the `ban_list` validator when it is " + "given this list's `ban_list_id`." + ), + }, + "domain": { + "type": "string", + "description": ( + "Caller-defined grouping label (e.g. 'abuse'). Used to filter " + "lists on GET /guardrails/ban_lists." + ), + }, + "is_public": { + "type": "boolean", + "default": False, + "description": ( + "When true the list is readable by other tenants. Updates and " + "deletes remain restricted to the owning project." + ), + }, +} + +_LLM_PROMPT_CONFIG_FIELDS: dict[str, Any] = { + "validator_name": { + "type": "string", + "description": ( + "Which LLM-backed validator this prompt drives (e.g. " + "`topic_relevance`, `answer_relevance_custom_llm`). Immutable " + "after creation. Call GET /api/v1/guardrails for the current set." + ), + }, + "name": {"type": "string", "description": "Human-readable name."}, + "description": {"type": "string", "description": "What this prompt evaluates."}, + "prompt_schema_version": { + "type": "integer", + "default": 1, + "description": ( + "Version of the prompt contract, for callers that iterate on " + "wording. `topic_relevance_llm` reads it as: 1 = the prompt lists " + "allowed topics, 2 = forbidden topics, 3 = both." + ), + }, + "llm_prompt": { + "type": "string", + "description": ( + "The prompt text. For `answer_relevance_custom_llm` this must " + "contain both the `{query}` and `{answer}` placeholders; the " + "guardrails service rejects it with a 422 otherwise." + ), + }, +} + +_VALIDATOR_CONFIG_FIELDS: dict[str, Any] = { + "name": { + "type": "string", + "description": ( + "Unique name for this config within the project. Uniqueness is " + "enforced on the name alone, so the same validator type may be " + "registered more than once under different names." + ), + }, + "type": { + "type": "string", + "description": ( + "Which validator implementation to run. Call GET /api/v1/guardrails " + "for the types this deployment supports and the tuning keys each " + "one accepts." + ), + }, + "stage": { + "type": "string", + "enum": ["input", "output"], + "description": ( + "Which text this config is intended for. Advisory only at run " + "time: POST /guardrails routes on the text it is given, not on " + "this field." + ), + }, + "on_fail_action": { + "type": "string", + "enum": ["exception", "fix", "rephrase"], + "default": "fix", + "description": ( + "Action taken when this validator fails. `fix` falls back to an " + "empty string for validators with no programmatic repair." + ), + }, + "is_enabled": { + "type": "boolean", + "default": True, + "description": "Whether this config is eligible to run.", + }, +} + +_IDENTITY_FIELDS: dict[str, Any] = { + "id": {"type": "string", "format": "uuid"}, + "created_at": {"type": "string", "format": "date-time"}, + "updated_at": {"type": "string", "format": "date-time"}, +} + + +def _optional(fields: dict[str, Any]) -> dict[str, Any]: + """Same fields, none required — the PATCH shape. Omitted keys are + forwarded as omitted, so they leave stored values untouched.""" + return {key: {**value, "nullable": True} for key, value in fields.items()} + + +_BAN_LIST = { + "type": "object", + "description": "A ban list as returned by the guardrails service.", + "properties": {**_BAN_LIST_FIELDS, **_IDENTITY_FIELDS}, +} + +_LLM_PROMPT_CONFIG = { + "type": "object", + "description": "An LLM prompt config as returned by the guardrails service.", + "properties": { + **_LLM_PROMPT_CONFIG_FIELDS, + "is_active": {"type": "boolean"}, + **_IDENTITY_FIELDS, + }, +} + +_VALIDATOR_CONFIG = { + "type": "object", + "description": ( + "A validator config as returned by the guardrails service. The stored " + "row is flattened together with its JSONB tuning config, so responses " + "carry validator-specific keys beyond the ones listed here." + ), + "properties": { + **_VALIDATOR_CONFIG_FIELDS, + "organization_id": {"type": "integer"}, + "project_id": {"type": "integer"}, + **_IDENTITY_FIELDS, + }, + "additionalProperties": True, +} + +GUARDRAILS_COMPONENT_SCHEMAS: dict[str, dict[str, Any]] = { + "GuardrailsBanListCreate": { + "type": "object", + "properties": _BAN_LIST_FIELDS, + "required": ["name", "description", "banned_words", "domain"], + }, + "GuardrailsBanListUpdate": { + "type": "object", + "description": "Omitted fields are left unchanged.", + "properties": _optional(_BAN_LIST_FIELDS), + }, + "GuardrailsBanListResponse": _envelope(_BAN_LIST), + "GuardrailsBanListListResponse": _list_envelope(_BAN_LIST), + "GuardrailsLLMPromptConfigCreate": { + "type": "object", + "properties": _LLM_PROMPT_CONFIG_FIELDS, + "required": ["validator_name", "name", "description", "llm_prompt"], + }, + "GuardrailsLLMPromptConfigUpdate": { + "type": "object", + "description": ( + "`validator_name` is absent by design — it is immutable after " + "creation. `is_active` is settable only here; new configs are " + "created active." + ), + "properties": { + **_optional( + { + key: value + for key, value in _LLM_PROMPT_CONFIG_FIELDS.items() + if key != "validator_name" + } + ), + "is_active": {"type": "boolean", "nullable": True}, + }, + }, + "GuardrailsLLMPromptConfigResponse": _envelope(_LLM_PROMPT_CONFIG), + "GuardrailsLLMPromptConfigListResponse": _list_envelope(_LLM_PROMPT_CONFIG), + "GuardrailsValidatorConfigCreate": { + "type": "object", + "description": ( + "Extra keys are the validator's tuning config — e.g. " + "`entity_types` for `pii_remover`, `slur_severity` for " + "`content_safety`. Call GET /api/v1/guardrails for each type's " + "accepted keys. `organization_id`/`project_id` must not be sent: " + "the guardrails service derives the tenant from the authenticated " + "context and rejects those keys with a 422." + ), + "properties": _VALIDATOR_CONFIG_FIELDS, + "required": ["name", "type", "stage"], + "additionalProperties": True, + }, + "GuardrailsValidatorConfigUpdate": { + "type": "object", + "description": ( + "Only the base fields can be patched. The guardrails service " + "forbids extra keys here, so validator-specific tuning cannot be " + "changed through this route — delete and recreate the config " + "instead. Extras are forwarded rather than dropped, so the " + "upstream 422 is what the caller sees." + ), + "properties": _optional(_VALIDATOR_CONFIG_FIELDS), + "additionalProperties": True, + }, + "GuardrailsValidatorConfigResponse": _envelope(_VALIDATOR_CONFIG), + "GuardrailsValidatorConfigListResponse": _list_envelope(_VALIDATOR_CONFIG), + "GuardrailsValidatorTypeListResponse": { + "type": "object", + "description": ( + "The one guardrails route that is not wrapped in the standard " + "`{success, data, ...}` envelope; the service returns this bare " + "object." + ), + "properties": { + "validators": { + "type": "array", + "items": { + "type": "object", + "properties": { + "type": { + "type": "string", + "description": "Validator type discriminator.", + }, + "config": { + "type": "object", + "description": ( + "JSON Schema of the tuning fields this " + "validator accepts." + ), + }, + }, + }, + } + }, + }, + "GuardrailsDeleteResponse": _envelope( + { + "type": "object", + "properties": { + "message": { + "type": "string", + "description": "Human-readable confirmation.", + } + }, + } + ), + "GuardrailsErrorResponse": _envelope({"nullable": True}), + # Failures raised by this gateway go through `app.utils.APIResponse`, which + # carries an extra `errors` list the guardrails service's envelope has not. + "GuardrailsGatewayErrorResponse": { + **_envelope({"nullable": True}), + "properties": { + **_envelope({"nullable": True})["properties"], + "errors": { + "type": "array", + "nullable": True, + "items": { + "type": "object", + "properties": { + "field": {"type": "string"}, + "message": {"type": "string"}, + }, + }, + }, + }, + }, +} + +# ---------- validator catalogue ---------- +# +# One entry per validator type the guardrails service exposes, mirroring what +# `GET /api/v1/guardrails` returns at run time. Verified against +# `kaapi-guardrails` validator configs; that service stays the authority, so +# when the two disagree, believe the live response and fix this table. + + +class ConfigKey(NamedTuple): + """One tuning key of a validator type. + + `schema` is what the live response carries for this key, trimmed of + JSON Schema bookkeeping (`title`, `$defs`, `anyOf` wrappers around + optionals). `description` is for humans and is not part of the response. + """ + + name: str + schema: dict[str, Any] + description: str + + +class ValidatorType(NamedTuple): + type: str + summary: str + runs_on: str + config: tuple[ConfigKey, ...] = () + notes: tuple[str, ...] = () + + +class ValidatorGroup(NamedTuple): + """A themed run of validator types, in the order they should be read.""" + + title: str + blurb: str + types: tuple[str, ...] + + +# Three questions a reader actually arrives with, rather than one flat list of +# twelve. Each group leads with the type to reach for first. +GROUPS: tuple[ValidatorGroup, ...] = ( + ValidatorGroup( + "Abuse, profanity and unsafe content", + "Start with `content_safety`. The rest are narrower checks to add " + "when it is not enough on its own.", + ( + "content_safety", + "uli_slur_match", + "profanity_free", + "nsfw_text", + "llamaguard_7b", + "gender_assumption_bias", + ), + ), + ValidatorGroup( + "Personal data and banned words", + "Both redact rather than block, so the conversation carries on with " + "the sensitive words removed.", + ("pii_remover", "ban_list"), + ), + ValidatorGroup( + "Staying on topic, and answer quality", + "All four call an LLM, so each adds a little latency per message. " + "Reach for `topic_relevance_llm` first.", + ( + "topic_relevance_llm", + "topic_relevance", + "answer_relevance_custom_llm", + "llm_critic", + ), + ), +) + + +_SEVERITY_LEVELS = ["low", "medium", "high", "all"] +_PII_ENTITY_TYPES = [ + "CREDIT_CARD", + "EMAIL_ADDRESS", + "IBAN_CODE", + "IP_ADDRESS", + "LOCATION", + "MEDICAL_LICENSE", + "NRP", + "PERSON", + "PHONE_NUMBER", + "URL", + "IN_AADHAAR", + "IN_PAN", + "IN_PASSPORT", + "IN_VEHICLE_REGISTRATION", + "IN_VOTER", +] +_PII_DEFAULT_ENTITY_TYPES = [ + entity + for entity in _PII_ENTITY_TYPES + if entity not in ("LOCATION", "NRP", "PERSON") +] +_NEEDS_OPENAI_KEY = ( + "Needs an OpenAI key on the guardrails service. Without one the validator " + "fails the request with an explicit error rather than passing silently." +) +_NO_PARTIAL_FIX = ( + "There is no partial repair: with `on_fail_action: fix` a failure returns " + "an empty `safe_text`, and `metadata.reason` names this validator." +) + +VALIDATOR_TYPES: tuple[ValidatorType, ...] = ( + ValidatorType( + type="uli_slur_match", + summary=( + "Matches a curated Hindi/English slur list and redacts each hit " + "with `[REDACTED_SLUR]`, leaving the rest of the message intact." + ), + runs_on="input and output", + config=( + ConfigKey( + "languages", + {"type": "array", "items": {"type": "string"}, "default": ["en", "hi"]}, + "Slur lists to load. `en` and `hi` are the curated ones.", + ), + ConfigKey( + "severity", + {"type": "string", "enum": _SEVERITY_LEVELS, "default": "all"}, + "Lowest severity to act on. `all` catches everything.", + ), + ), + ), + ValidatorType( + type="profanity_free", + summary="Flags profanity anywhere in the text.", + runs_on="input and output", + notes=( + _NO_PARTIAL_FIX, + "If you want slurs redacted *and* profanity checked, use " + "`content_safety` instead of running both — it orders them so a " + "redacted slur does not then trip the profanity check.", + ), + ), + ValidatorType( + type="content_safety", + summary=( + "Slur redaction followed by a profanity check, as one validator. " + "The slur pass runs first, so text it has already cleaned is not " + "blanked by the profanity pass." + ), + runs_on="input and output", + config=( + ConfigKey( + "slur_severity", + {"type": "string", "enum": _SEVERITY_LEVELS, "default": "all"}, + "Lowest slur severity to redact.", + ), + ConfigKey( + "profanity_threshold", + {"type": "number", "minimum": 0, "maximum": 1, "default": 0.9}, + ( + "Profanity probability at or above which the text fails " + "and is blanked. Lower it to be stricter." + ), + ), + ), + ), + ValidatorType( + type="pii_remover", + summary=("Replaces personal data with placeholders such as ``."), + runs_on="input and output", + config=( + ConfigKey( + "entity_types", + { + "type": "array", + "items": {"type": "string", "enum": _PII_ENTITY_TYPES}, + "default": _PII_DEFAULT_ENTITY_TYPES, + }, + ( + "Which entities to mask. Anything outside the listed " + "values is rejected when the config is saved." + ), + ), + ), + notes=( + "`PERSON`, `LOCATION` and `NRP` are left out of the default on " + "purpose: they come from an English-only name model that misfires " + "on Indic names. Add them only if you have checked the result on " + "your own traffic. Every other entity is pattern-matched.", + ), + ), + ValidatorType( + type="ban_list", + summary="Redacts words you supply — product names, competitors, anything.", + runs_on="input and output", + config=( + ConfigKey( + "banned_words", + {"type": "array", "items": {"type": "string"}, "default": None}, + "The words, inline.", + ), + ConfigKey( + "ban_list_id", + {"type": "string", "format": "uuid", "default": None}, + ( + "A list stored through `/guardrails/ban_lists`, resolved " + "to its words at run time." + ), + ), + ), + notes=( + "Give exactly one of the two. A config with neither is rejected. " + "`ban_list_id` is the maintainable choice: edit the list once and " + "every validator using it follows.", + ), + ), + ValidatorType( + type="gender_assumption_bias", + summary=( + "Flags text that assumes someone's gender — 'ask her doctor' when " + "nothing said the patient is a woman." + ), + runs_on="input and output", + config=( + ConfigKey( + "categories", + { + "type": "array", + "items": { + "type": "string", + "enum": ["generic", "healthcare", "education", "all"], + }, + "default": ["all"], + }, + "Which domains to check. `all` enables every category.", + ), + ), + ), + ValidatorType( + type="nsfw_text", + summary="Classifies sexual/explicit content with a local model.", + runs_on="input and output", + config=( + ConfigKey( + "threshold", + {"type": "number", "default": 0.8}, + "Probability above which text counts as NSFW.", + ), + ConfigKey( + "validation_method", + {"type": "string", "default": "sentence"}, + ( + "`sentence` scores each sentence and fails if any one " + "crosses the threshold; `full` scores the message as a " + "whole, which is faster on short messages." + ), + ), + ConfigKey( + "device", + {"type": "string", "default": "cpu"}, + "`cpu` or `cuda`.", + ), + ConfigKey( + "model_name", + {"type": "string", "default": "michellejieli/NSFW_text_classifier"}, + ( + "HuggingFace classifier. It must emit an `NSFW` label — a " + "model that labels its classes differently never fails, " + "whatever it scores." + ), + ), + ), + notes=( + _NO_PARTIAL_FIX, + "Runs a transformer locally: budget roughly 200–500 ms per " + "request on CPU. The default model is English-focused.", + ), + ), + ValidatorType( + type="llamaguard_7b", + summary=( + "Classifies text as safe or unsafe against LlamaGuard's policy " + "set (violence and hate, sexual content, criminal planning, " + "weapons, drugs, self-harm)." + ), + runs_on="input and output", + config=( + ConfigKey( + "policies", + {"type": "array", "items": {"type": "string"}, "default": None}, + ( + "Subset to enforce, from `no_violence_hate`, " + "`no_sexual_content`, `no_criminal_planning`, " + "`no_guns_and_illegal_weapons`, `no_illegal_drugs`, " + "`no_encourage_self_harm`. Omit for all of them." + ), + ), + ), + notes=( + "Runs remotely on the Guardrails Hub, so the guardrails service " + "must have remote inferencing enabled.", + _NO_PARTIAL_FIX, + "Expect false positives on news, clinical and legal text, which " + "discuss these topics without endorsing them.", + ), + ), + ValidatorType( + type="llm_critic", + summary=( + "Scores the text against metrics you write, using an LLM as the " + "judge. The escape hatch for checks no other validator covers." + ), + runs_on="input or output", + config=( + ConfigKey( + "metrics", + {"type": "object"}, + ( + "Metric name to a plain-English description of " + 'what earns a high score, e.g. `{"clarity": "Is the ' + 'answer easy to follow?"}`.' + ), + ), + ConfigKey( + "max_score", + {"type": "integer"}, + "Top of the scoring scale, e.g. `5`.", + ), + ConfigKey( + "llm_callable", + {"type": "string"}, + "Judge model, e.g. `gpt-4o-mini`.", + ), + ), + notes=( + "All three keys are required — unlike every other type, there is " + "no default to fall back on.", + _NEEDS_OPENAI_KEY, + "LLM judges vary run to run. Keep `max_score` small and read a " + "sample of verdicts before trusting it in production.", + ), + ), + ValidatorType( + type="topic_relevance_llm", + summary=( + "Keeps the conversation inside your subject with one LLM call, " + "scoring the message 1–3 and failing below `threshold`." + ), + runs_on="input", + config=( + ConfigKey( + "configuration", + {"type": "string", "default": None}, + ( + "The topic text itself. Read as *forbidden* topics by " + "default — see `prompt_schema_version`." + ), + ), + ConfigKey( + "prompt_schema_version", + {"type": "integer", "minimum": 1, "default": 2}, + ( + "How `configuration` is read: `1` = the topics you allow, " + "`2` = the topics you forbid, `3` = both (forbidden wins)." + ), + ), + ConfigKey( + "threshold", + {"type": "integer", "minimum": 1, "maximum": 3, "default": 2}, + ( + "Minimum score to pass. `2` lets ambiguous messages " + "through; `3` blocks them too." + ), + ), + ConfigKey( + "fallback_message", + {"type": "string", "minLength": 1, "default": None}, + ( + "What to say when the message is off-topic. The same LLM " + "call translates it into the user's language, so a Hindi " + "message gets a Hindi reply. Only used with " + "`on_fail_action: rephrase`." + ), + ), + ConfigKey( + "topic_relevance_config_id", + {"type": "string", "format": "uuid", "default": None}, + ( + "Use a prompt stored through `/guardrails/llm_prompt_configs` " + "instead of inline `configuration`." + ), + ), + ConfigKey( + "llm_callable", + {"type": "string", "default": "gpt-4o-mini"}, + "Model that does the scoring.", + ), + ), + notes=( + "Give either `configuration` or `topic_relevance_config_id`.", + "Pair it with `on_fail_action: rephrase`. With `fix` the user " + "gets an empty string, since there is nothing to repair.", + _NEEDS_OPENAI_KEY, + ), + ), + ValidatorType( + type="topic_relevance", + summary=( + "The older topic check, scored by `llm_critic` under the hood. " + "Prefer `topic_relevance_llm` for new work: one call, a clearer " + "score, and a localized fallback reply." + ), + runs_on="input", + config=( + ConfigKey( + "topic_relevance_config_id", + {"type": "string", "format": "uuid", "default": None}, + ( + "Required. A prompt stored through " + "`/guardrails/llm_prompt_configs`; its text and schema " + "version are loaded at run time." + ), + ), + ConfigKey( + "configuration", + {"type": "string", "default": None}, + "Filled in from the stored prompt; you do not send it.", + ), + ConfigKey( + "prompt_schema_version", + {"type": "integer", "default": None}, + "Also filled in from the stored prompt.", + ), + ConfigKey( + "llm_callable", + {"type": "string", "default": "gpt-4o-mini"}, + "Model that does the scoring.", + ), + ), + notes=( + "The stored prompt must contain the `{{TOPIC_CONFIGURATION}}` " + "placeholder.", + _NEEDS_OPENAI_KEY, + ), + ), + ValidatorType( + type="answer_relevance_custom_llm", + summary=( + "Judges whether an answer actually addresses the question that " + "was asked." + ), + runs_on="output", + config=( + ConfigKey( + "custom_prompt_id", + {"type": "string", "format": "uuid", "default": None}, + ( + "A scoring prompt stored through " + "`/guardrails/llm_prompt_configs`." + ), + ), + ConfigKey( + "prompt_template", + {"type": "string", "default": None}, + ( + "The same thing inline. Must contain the `{query}` and " + "`{answer}` placeholders." + ), + ), + ConfigKey( + "llm_callable", + {"type": "string", "default": "gpt-4o-mini"}, + "Model that does the judging.", + ), + ConfigKey( + "input", + {"type": "string", "default": ""}, + ( + "Filled in by the service from the guardrails request. " + "Listed because it appears in the schema; do not send it." + ), + ), + ConfigKey( + "output", + {"type": "string", "default": ""}, + "Filled in by the service too. Do not send it.", + ), + ), + notes=( + "Give either `custom_prompt_id` or `prompt_template`.", + "This validator reads the question and the answer together, so a " + "guardrails run using it must send `input` as a JSON string: " + '`{"query": "...", "answer": "..."}`.', + _NEEDS_OPENAI_KEY, + ), + ), +) + + +def _type_label(schema: dict[str, Any]) -> str: + if "enum" in schema: + return "one of " + ", ".join(f"`{value}`" for value in schema["enum"]) + if schema.get("type") == "array": + items = schema.get("items", {}) + if "enum" in items: + return "array of " + ", ".join(f"`{value}`" for value in items["enum"]) + return f"array of {items.get('type', 'any')}" + if schema.get("format") == "uuid": + return "uuid" + label = schema.get("type", "any") + bounds = [ + f"{name} {schema[key]}" + for key, name in ( + ("minimum", "min"), + ("maximum", "max"), + ("minLength", "min length"), + ) + if key in schema + ] + return f"{label} ({', '.join(bounds)})" if bounds else label + + +def _default_label(schema: dict[str, Any]) -> str: + if "default" not in schema: + return "**required**" + default = schema["default"] + if default is None: + return "unset" + if isinstance(default, list) and len(default) > 4: + return f"all {len(default)} pattern-based types" + return f"`{json.dumps(default)}`" + + +def _by_type() -> dict[str, ValidatorType]: + catalogued = {validator.type: validator for validator in VALIDATOR_TYPES} + grouped = {name for group in GROUPS for name in group.types} + # A type missing from GROUPS would silently vanish from the docs, which is + # worse than a loud failure at import. + if catalogued.keys() != grouped: + raise RuntimeError( + "VALIDATOR_TYPES and GROUPS disagree: " + f"{sorted(catalogued.keys() ^ grouped)}" + ) + return catalogued + + +def _ordered_types() -> list[ValidatorType]: + """Catalogue in reading order: grouped, groups in `GROUPS` order.""" + catalogued = _by_type() + return [catalogued[name] for group in GROUPS for name in group.types] + + +def _render_validator(validator: ValidatorType) -> list[str]: + lines = [ + f"#### `{validator.type}`", + "", + f"{validator.summary} Runs on **{validator.runs_on}**.", + "", + ] + if validator.config: + lines += ["| Key | Type | Default | What it does |", "|---|---|---|---|"] + lines += [ + f"| `{key.name}` | {_type_label(key.schema)} | " + f"{_default_label(key.schema)} | {key.description} |" + for key in validator.config + ] + else: + lines.append( + "No tuning keys of its own — the shared fields are the whole config." + ) + if validator.notes: + lines.append("") + lines += [f"- {note}" for note in validator.notes] + return lines + + +def render_validator_reference() -> str: + """Render the catalogue as Markdown, appended to the route description. + + Generated rather than hand-written so the reference and the example + response in this module cannot drift apart. + """ + lines = [ + "", + "
", + "", + "## The validator types", + "", + "Twelve of them, grouped by the problem they solve. Every one also " + "takes the shared fields above.", + ] + catalogued = _by_type() + for group in GROUPS: + lines += ["", "
", "", f"### {group.title}", "", group.blurb] + for name in group.types: + lines += ["", "---", ""] + lines += _render_validator(catalogued[name]) + return "\n".join(lines) + "\n" + + +def _validator_catalogue_example() -> dict[str, Any]: + """The catalogue in the shape `GET /api/v1/guardrails` returns it.""" + return { + "validators": [ + { + "type": validator.type, + "config": { + "type": "object", + "properties": {key.name: key.schema for key in validator.config}, + }, + } + for validator in _ordered_types() + ] + } + + +# ---------- examples ---------- + +BAN_LIST_CREATE_EXAMPLE = { + "name": "Safety Banned Terms", + "description": "Terms not allowed under this tenant's policy", + "banned_words": ["scam", "fraud"], + "domain": "abuse", + "is_public": False, +} +BAN_LIST_UPDATE_EXAMPLE = {"banned_words": ["scam", "fraud", "phishing"]} +_BAN_LIST_EXAMPLE = { + **BAN_LIST_CREATE_EXAMPLE, + "id": _UUID_EXAMPLE, + "created_at": _TIMESTAMP_EXAMPLE, + "updated_at": _TIMESTAMP_EXAMPLE, +} + +LLM_PROMPT_CONFIG_CREATE_EXAMPLES: dict[str, tuple[str, Any]] = { + "topic_relevance": ( + "Topic scope prompt", + { + "validator_name": "topic_relevance", + "name": "Maternal Health Scope", + "description": "Topic guard for a maternal health support bot", + "prompt_schema_version": 2, + "llm_prompt": "Forbidden topics: politics, religion, sports.", + }, + ), + "answer_relevance_custom_llm": ( + "Scoring prompt — must contain {query} and {answer}", + { + "validator_name": "answer_relevance_custom_llm", + "name": "Support Answer Grader", + "description": "Grades how well the answer addresses the query", + "prompt_schema_version": 1, + "llm_prompt": ( + "Query: {query}\nAnswer: {answer}\n" "Score the relevance from 1 to 5." + ), + }, + ), +} +LLM_PROMPT_CONFIG_UPDATE_EXAMPLE = { + "description": "Updated scope", + "is_active": True, +} +_LLM_PROMPT_CONFIG_EXAMPLE = { + "id": _UUID_EXAMPLE, + **LLM_PROMPT_CONFIG_CREATE_EXAMPLES["topic_relevance"][1], + "is_active": True, + "created_at": _TIMESTAMP_EXAMPLE, + "updated_at": _TIMESTAMP_EXAMPLE, +} + +VALIDATOR_CONFIG_CREATE_EXAMPLES: dict[str, tuple[str, Any]] = { + "pii_remover": ( + "PII masking — entity_types is validator-specific tuning", + { + "name": "PII Redaction Input", + "type": "pii_remover", + "stage": "input", + "on_fail_action": "fix", + "is_enabled": True, + "entity_types": ["EMAIL_ADDRESS", "PHONE_NUMBER", "IN_AADHAAR"], + }, + ), + "content_safety": ( + "Slur redaction plus profanity check", + { + "name": "Content Safety Input", + "type": "content_safety", + "stage": "input", + "on_fail_action": "fix", + "slur_severity": "all", + "profanity_threshold": 0.9, + }, + ), + "ban_list": ( + "Ban list by id, resolved upstream", + { + "name": "Abuse Terms Input", + "type": "ban_list", + "stage": "input", + "on_fail_action": "fix", + "ban_list_id": _UUID_EXAMPLE, + }, + ), + "topic_relevance_llm": ( + "Topic scope with a localized fallback reply", + { + "name": "Farming Scope Input", + "type": "topic_relevance_llm", + "stage": "input", + "on_fail_action": "rephrase", + "configuration": "politics, religion, sports", + "prompt_schema_version": 2, + "fallback_message": "I can only help with farming questions.", + }, + ), +} +VALIDATOR_CONFIG_UPDATE_EXAMPLE = {"is_enabled": False, "on_fail_action": "rephrase"} +_VALIDATOR_CONFIG_EXAMPLE = { + "id": _UUID_EXAMPLE, + "organization_id": 1, + "project_id": 1, + **VALIDATOR_CONFIG_CREATE_EXAMPLES["pii_remover"][1], + "created_at": _TIMESTAMP_EXAMPLE, + "updated_at": _TIMESTAMP_EXAMPLE, +} + + +def _success(data: Any) -> dict[str, Any]: + return {"success": True, "data": data, "error": None, "metadata": None} + + +def _failure(message: str) -> dict[str, Any]: + """Failure body as the guardrails service writes it, echoed verbatim.""" + return {"success": False, "data": None, "error": message, "metadata": None} + + +def _gateway_failure(message: str) -> dict[str, Any]: + """Failure body as this gateway writes it; `errors` is only populated for + request-validation failures.""" + return {**_failure(message), "errors": None} + + +# ---------- documented responses ---------- + +# Gateway-side failures: the request never reaches the guardrails service. +UNAUTHORIZED = ResponseDoc( + "Missing or invalid API key.", + _gateway_failure("Not authenticated"), + ref("GuardrailsGatewayErrorResponse"), +) +FORBIDDEN = ResponseDoc( + "The API key's project is not permitted to use guardrails.", + _gateway_failure("Permission denied"), + ref("GuardrailsGatewayErrorResponse"), +) +BAD_GATEWAY = ResponseDoc( + "The guardrails service was unreachable or returned a non-JSON body.", + _gateway_failure("Guardrails service unavailable"), + ref("GuardrailsGatewayErrorResponse"), +) +JOB_NOT_FOUND = ResponseDoc( + "No guardrails job with this id in your project. Also returned for a job " + "of another type, so job ids cannot be probed across endpoints.", + _gateway_failure("Job not found"), + ref("GuardrailsGatewayErrorResponse"), +) + +# The async job routes run locally, so only the gateway's own auth failures +# apply; their 200 schemas come from the Pydantic models they still validate. +JOB_RESPONSES = {401: UNAUTHORIZED, 403: FORBIDDEN} +JOB_ITEM_RESPONSES = {**JOB_RESPONSES, 404: JOB_NOT_FOUND} +# Echoed verbatim from the guardrails service, which owns validation. +UPSTREAM_NOT_FOUND = ResponseDoc( + "No such record in this project (echoed from the guardrails service).", + _failure("Not found"), + ref("GuardrailsErrorResponse"), +) +UPSTREAM_VALIDATION_ERROR = ResponseDoc( + "Rejected by the guardrails service, which owns validation for these " + "payloads. The gateway forwards bodies unchanged.", + _failure("Missing required field(s): name"), + ref("GuardrailsErrorResponse"), +) + +PROXY_READ_RESPONSES = { + 401: UNAUTHORIZED, + 403: FORBIDDEN, + 502: BAD_GATEWAY, +} +PROXY_ITEM_RESPONSES = {**PROXY_READ_RESPONSES, 404: UPSTREAM_NOT_FOUND} +PROXY_WRITE_RESPONSES = { + **PROXY_READ_RESPONSES, + 422: UPSTREAM_VALIDATION_ERROR, +} +PROXY_ITEM_WRITE_RESPONSES = {**PROXY_ITEM_RESPONSES, 422: UPSTREAM_VALIDATION_ERROR} + + +def _ok(description: str, example: Any, schema_name: str) -> ResponseDoc: + return ResponseDoc(description, example, ref(schema_name)) + + +BAN_LIST_OK = _ok( + "The ban list.", _success(_BAN_LIST_EXAMPLE), "GuardrailsBanListResponse" +) +BAN_LIST_LIST_OK = _ok( + "Ban lists visible to this project.", + _success([_BAN_LIST_EXAMPLE]), + "GuardrailsBanListListResponse", +) +LLM_PROMPT_CONFIG_OK = _ok( + "The LLM prompt config.", + _success(_LLM_PROMPT_CONFIG_EXAMPLE), + "GuardrailsLLMPromptConfigResponse", +) +LLM_PROMPT_CONFIG_LIST_OK = _ok( + "LLM prompt configs for this project.", + _success([_LLM_PROMPT_CONFIG_EXAMPLE]), + "GuardrailsLLMPromptConfigListResponse", +) +VALIDATOR_CONFIG_OK = _ok( + "The validator config, flattened with its tuning keys.", + _success(_VALIDATOR_CONFIG_EXAMPLE), + "GuardrailsValidatorConfigResponse", +) +VALIDATOR_CONFIG_LIST_OK = _ok( + "Validator configs for this project.", + _success([_VALIDATOR_CONFIG_EXAMPLE]), + "GuardrailsValidatorConfigListResponse", +) +VALIDATOR_TYPE_LIST_OK = _ok( + "Every validator type this deployment supports, with the JSON Schema of " + "its tuning keys. Trimmed here for readability: the live response also " + "carries JSON Schema bookkeeping (`title`, `$defs`, `anyOf` wrappers " + "around optional fields) and the shared `type`, `stage` and `on_fail` " + "fields on every entry.", + _validator_catalogue_example(), + "GuardrailsValidatorTypeListResponse", +) +DELETE_OK = _ok( + "Deleted. The guardrails service answers deletes with a confirmation " + "body rather than a 204.", + _success({"message": "Ban list deleted successfully"}), + "GuardrailsDeleteResponse", +) diff --git a/backend/app/api/docs/openapi_config.py b/backend/app/api/docs/openapi_config.py index 99cee7875..317f05d28 100644 --- a/backend/app/api/docs/openapi_config.py +++ b/backend/app/api/docs/openapi_config.py @@ -5,6 +5,14 @@ for organizing and enhancing the API documentation. """ +from app.api.docs.guardrails_schemas import GUARDRAILS_COMPONENT_SCHEMAS +from app.api.openapi_docs import inject_component_schemas, register_component_schemas + +# Schemas for routes that document a contract they do not validate (the +# guardrails proxy). Registered at import so they are in `components` before +# any `$ref` to them is resolved. +register_component_schemas(GUARDRAILS_COMPONENT_SCHEMAS) + # Tag metadata for organizing endpoints in documentation tags_metadata = [ { @@ -27,6 +35,13 @@ "name": "LLM", "description": "Large Language Model inference and interaction endpoints", }, + { + "name": "Guardrails", + "description": ( + "Text safety validation: applying guardrails to text, and managing " + "the validator configs, ban lists and LLM prompts they run on" + ), + }, { "name": "Evaluation", "description": "Dataset upload, running evaluations, listing datasets as well as evaluations", @@ -89,6 +104,7 @@ "Collections", "Config Management", "LLM", + "Guardrails", "Evaluation", "Fine Tuning", "Model Evaluation", @@ -122,6 +138,7 @@ def customize_openapi_schema(openapi_schema: dict) -> dict: Returns: The customized OpenAPI schema with x-tagGroups and other extensions """ + openapi_schema = inject_component_schemas(openapi_schema) openapi_schema["x-tagGroups"] = tag_groups deprecated_tags = ["Assistants", "Threads"] diff --git a/backend/app/api/openapi_docs.py b/backend/app/api/openapi_docs.py new file mode 100644 index 000000000..0a60c74f1 --- /dev/null +++ b/backend/app/api/openapi_docs.py @@ -0,0 +1,126 @@ +"""Docs-only OpenAPI metadata for pass-through routes. + +Nothing in this module validates anything. `openapi_extra` and `responses=` +only shape what Swagger/ReDoc render, which is what a proxy route needs: the +downstream service owns validation, so mirroring its schemas as Pydantic +models here would duplicate rules that drift the moment downstream changes. + +Schemas registered through `register_component_schemas` are injected into +`components.schemas` by `customize_openapi_schema`, so `openapi_extra` can +`$ref` them and Swagger still renders a typed body with no local model. +""" + +from typing import Any, NamedTuple + +JSON_MEDIA_TYPE = "application/json" + +_REGISTERED_COMPONENTS: dict[str, dict[str, Any]] = {} + + +class ResponseDoc(NamedTuple): + """One documented status code: prose, an example body, optional schema.""" + + description: str + example: Any + schema: dict[str, Any] | None = None + + +def register_component_schemas(schemas: dict[str, dict[str, Any]]) -> None: + """Publish hand-written JSON Schemas under `components.schemas`.""" + _REGISTERED_COMPONENTS.update(schemas) + + +def inject_component_schemas(openapi_schema: dict[str, Any]) -> dict[str, Any]: + """Merge registered schemas into a generated spec. + + Generated components win on a name clash: a real Pydantic model is always + a better description of a route than a hand-written stand-in. + """ + components = openapi_schema.setdefault("components", {}).setdefault("schemas", {}) + for name, schema in _REGISTERED_COMPONENTS.items(): + components.setdefault(name, schema) + return openapi_schema + + +def ref(name: str) -> dict[str, str]: + """A `$ref` to a component schema, registered or FastAPI-generated.""" + return {"$ref": f"#/components/schemas/{name}"} + + +def infer_schema(value: Any) -> dict[str, Any]: + """Best-effort JSON Schema from an example value.""" + if isinstance(value, bool): + return {"type": "boolean"} + if isinstance(value, int): + return {"type": "integer"} + if isinstance(value, float): + return {"type": "number"} + if isinstance(value, str): + return {"type": "string"} + if value is None: + return {"nullable": True} + if isinstance(value, list): + return {"type": "array", "items": infer_schema(value[0]) if value else {}} + if isinstance(value, dict): + return { + "type": "object", + "properties": {key: infer_schema(item) for key, item in value.items()}, + } + return {} + + +def doc( + *, + request_schema: dict[str, Any] | None = None, + request: Any = None, + request_examples: dict[str, tuple[str, Any]] | None = None, + required: bool = True, + responses: dict[int, ResponseDoc] | None = None, +) -> dict[str, Any]: + """Build decorator kwargs: `@router.post("/x", **doc(...))`. + + `request_examples` maps a name to `(summary, body)` and renders as + Swagger's example dropdown; `request` is the single-example shorthand. + Pass `request_schema=ref(...)` to keep a typed body in the docs without a + local Pydantic model. + """ + kwargs: dict[str, Any] = {} + + if request_schema is not None or request is not None or request_examples: + # With only `request_examples`, infer from the first one — inferring + # from `request` (None) would document the body as nullable. + inferred_from = request + if inferred_from is None and request_examples: + inferred_from = next(iter(request_examples.values()))[1] + content: dict[str, Any] = { + "schema": request_schema or infer_schema(inferred_from) + } + if request_examples: + content["examples"] = { + name: {"summary": summary, "value": body} + for name, (summary, body) in request_examples.items() + } + elif request is not None: + content["example"] = request + kwargs["openapi_extra"] = { + "requestBody": { + "required": required, + "content": {JSON_MEDIA_TYPE: content}, + } + } + + if responses: + kwargs["responses"] = { + status: { + "description": response.description, + "content": { + JSON_MEDIA_TYPE: { + "schema": response.schema or infer_schema(response.example), + "example": response.example, + } + }, + } + for status, response in responses.items() + } + + return kwargs diff --git a/backend/app/api/routes/guardrails.py b/backend/app/api/routes/guardrails.py index d3052ee88..9d0ee55ce 100644 --- a/backend/app/api/routes/guardrails.py +++ b/backend/app/api/routes/guardrails.py @@ -2,11 +2,35 @@ from typing import Annotated, Any from uuid import UUID -from fastapi import APIRouter, Depends, HTTPException, Query, Response +from fastapi import APIRouter, Body, Depends, HTTPException, Path, Query, Response from fastapi.responses import JSONResponse from opentelemetry import trace from app.api.deps import AuthContextDep, SessionDep +from app.api.docs.guardrails_schemas import ( + BAN_LIST_CREATE_EXAMPLE, + BAN_LIST_LIST_OK, + BAN_LIST_OK, + BAN_LIST_UPDATE_EXAMPLE, + DELETE_OK, + JOB_ITEM_RESPONSES, + JOB_RESPONSES, + LLM_PROMPT_CONFIG_CREATE_EXAMPLES, + LLM_PROMPT_CONFIG_LIST_OK, + LLM_PROMPT_CONFIG_OK, + LLM_PROMPT_CONFIG_UPDATE_EXAMPLE, + PROXY_ITEM_RESPONSES, + PROXY_ITEM_WRITE_RESPONSES, + PROXY_READ_RESPONSES, + PROXY_WRITE_RESPONSES, + VALIDATOR_CONFIG_CREATE_EXAMPLES, + VALIDATOR_CONFIG_LIST_OK, + VALIDATOR_CONFIG_OK, + VALIDATOR_CONFIG_UPDATE_EXAMPLE, + VALIDATOR_TYPE_LIST_OK, + render_validator_reference, +) +from app.api.openapi_docs import doc, ref from app.api.permissions import Permission, require_permission from app.core.rate_monitor import monitor_rate from app.core.telemetry import log_context @@ -27,9 +51,44 @@ router = APIRouter(tags=["Guardrails"]) guardrails_callback_router = APIRouter() +ProxyBody = Annotated[ + Any, + Body( + description=( + "Forwarded to the guardrails service unchanged. That service owns " + "validation for this payload, so the gateway neither checks nor " + "reshapes it; its 422 is what you see on a bad body." + ) + ), +] +"""Body of a management-API proxy route. + +Deliberately untyped at run time: a Pydantic mirror of the upstream schema +would reject payloads upstream accepts the moment the two drift (a new +validator type, a new tuning key). The documented shape lives in +`app.api.docs.guardrails_schemas` instead. +""" + + +@guardrails_callback_router.post( + "{$callback_url}", + name="guardrails_callback", +) +def guardrails_callback_notification( + body: APIResponse[GuardrailsCallbackData], +) -> None: + """Callback delivered to `callback_url` when a guardrails job finishes. + + On success `success=True` and `data` carries the sanitised text. On a hard + block `success=False`, `error` carries the upstream message and `data` is + null. `metadata` echoes the request's `request_metadata` plus a + server-managed `warnings` list. + """ + @router.post( "/guardrails", + summary="Apply guardrails to text", description=load_description("guardrails/apply_guardrails.md"), response_model=APIResponse[GuardrailsJobImmediatePublic], callbacks=guardrails_callback_router.routes, @@ -37,6 +96,7 @@ Depends(require_permission(Permission.REQUIRE_PROJECT)), Depends(monitor_rate("llm_call")), ], + **doc(responses=JOB_RESPONSES), ) def apply_guardrails_endpoint( _current_user: AuthContextDep, @@ -111,7 +171,13 @@ def _upstream_response(status_code: int, payload: Any) -> Response: @router.get( "/guardrails", + summary="List validator types", + # The per-type reference is rendered from the same catalogue that builds + # the example response, so the two cannot disagree. + description=load_description("guardrails/list_validator_types.md") + + render_validator_reference(), dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], + **doc(responses={200: VALIDATOR_TYPE_LIST_OK, **PROXY_READ_RESPONSES}), ) def list_guardrails_validator_types(_current_user: AuthContextDep) -> Response: """List the validator types supported upstream and their JSON schemas.""" @@ -126,10 +192,17 @@ def list_guardrails_validator_types(_current_user: AuthContextDep) -> Response: @router.post( "/guardrails/ban_lists", + summary="Create a ban list", + description=load_description("guardrails/create_ban_list.md"), dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], + **doc( + request_schema=ref("GuardrailsBanListCreate"), + request=BAN_LIST_CREATE_EXAMPLE, + responses={200: BAN_LIST_OK, **PROXY_WRITE_RESPONSES}, + ), ) def create_guardrails_ban_list( - _current_user: AuthContextDep, body: dict[str, Any] + _current_user: AuthContextDep, body: ProxyBody = None ) -> Response: status_code, payload = proxy_guardrails_request( "POST", @@ -143,13 +216,22 @@ def create_guardrails_ban_list( @router.get( "/guardrails/ban_lists", + summary="List ban lists", + description=load_description("guardrails/list_ban_lists.md"), dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], + **doc(responses={200: BAN_LIST_LIST_OK, **PROXY_READ_RESPONSES}), ) def list_guardrails_ban_lists( _current_user: AuthContextDep, - domain: str | None = None, - offset: Annotated[int, Query(ge=0)] = 0, - limit: Annotated[int | None, Query(ge=1, le=100)] = None, + domain: Annotated[ + str | None, + Query(description="Filter to ban lists carrying this domain label."), + ] = None, + offset: Annotated[int, Query(ge=0, description="Rows to skip.")] = 0, + limit: Annotated[ + int | None, + Query(ge=1, le=100, description="Max rows to return. Unset means no limit."), + ] = None, ) -> Response: status_code, payload = proxy_guardrails_request( "GET", @@ -163,10 +245,17 @@ def list_guardrails_ban_lists( @router.post( "/guardrails/llm_prompt_configs", + summary="Create an LLM prompt config", + description=load_description("guardrails/create_llm_prompt_config.md"), dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], + **doc( + request_schema=ref("GuardrailsLLMPromptConfigCreate"), + request_examples=LLM_PROMPT_CONFIG_CREATE_EXAMPLES, + responses={200: LLM_PROMPT_CONFIG_OK, **PROXY_WRITE_RESPONSES}, + ), ) def create_guardrails_llm_prompt_config( - _current_user: AuthContextDep, body: dict[str, Any] + _current_user: AuthContextDep, body: ProxyBody = None ) -> Response: status_code, payload = proxy_guardrails_request( "POST", @@ -180,30 +269,55 @@ def create_guardrails_llm_prompt_config( @router.get( "/guardrails/llm_prompt_configs", + summary="List LLM prompt configs", + description=load_description("guardrails/list_llm_prompt_configs.md"), dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], + **doc(responses={200: LLM_PROMPT_CONFIG_LIST_OK, **PROXY_READ_RESPONSES}), ) def list_guardrails_llm_prompt_configs( _current_user: AuthContextDep, - validator_name: str | None = None, - offset: Annotated[int, Query(ge=0)] = 0, - limit: Annotated[int | None, Query(ge=1, le=100)] = None, + validator_name: Annotated[ + str | None, + Query( + description=( + "Filter to prompts driving this validator, e.g. " + "`topic_relevance` or `answer_relevance_custom_llm`." + ) + ), + ] = None, + offset: Annotated[int, Query(ge=0, description="Rows to skip.")] = 0, + limit: Annotated[ + int | None, + Query(ge=1, le=100, description="Max rows to return. Unset means no limit."), + ] = None, ) -> Response: status_code, payload = proxy_guardrails_request( "GET", "/llm_prompt_configs/", organization_id=_current_user.organization_.id, project_id=_current_user.project_.id, - params={"validator_name": validator_name, "offset": offset, "limit": limit}, + params={ + "validator_name": validator_name, + "offset": offset, + "limit": limit, + }, ) return _upstream_response(status_code, payload) @router.post( "/guardrails/validators/configs", + summary="Create a validator config", + description=load_description("guardrails/create_validator_config.md"), dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], + **doc( + request_schema=ref("GuardrailsValidatorConfigCreate"), + request_examples=VALIDATOR_CONFIG_CREATE_EXAMPLES, + responses={200: VALIDATOR_CONFIG_OK, **PROXY_WRITE_RESPONSES}, + ), ) def create_guardrails_validator_config( - _current_user: AuthContextDep, body: dict[str, Any] + _current_user: AuthContextDep, body: ProxyBody = None ) -> Response: status_code, payload = proxy_guardrails_request( "POST", @@ -217,13 +331,30 @@ def create_guardrails_validator_config( @router.get( "/guardrails/validators/configs", + summary="List validator configs", + description=load_description("guardrails/list_validator_configs.md"), dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], + **doc(responses={200: VALIDATOR_CONFIG_LIST_OK, **PROXY_READ_RESPONSES}), ) def list_guardrails_validator_configs( _current_user: AuthContextDep, - ids: Annotated[list[UUID] | None, Query()] = None, - stage: str | None = None, - type: str | None = None, + ids: Annotated[ + list[UUID] | None, + Query(description="Repeat to fetch several configs by id."), + ] = None, + stage: Annotated[ + str | None, + Query(description="Filter by the stage the config targets: input or output."), + ] = None, + type: Annotated[ + str | None, + Query( + description=( + "Filter by validator type. See GET /api/v1/guardrails for the " + "supported types." + ) + ), + ] = None, ) -> Response: status_code, payload = proxy_guardrails_request( "GET", @@ -241,10 +372,14 @@ def list_guardrails_validator_configs( @router.get( "/guardrails/validators/configs/{config_id}", + summary="Get a validator config", + description=load_description("guardrails/get_validator_config.md"), dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], + **doc(responses={200: VALIDATOR_CONFIG_OK, **PROXY_ITEM_RESPONSES}), ) def get_guardrails_validator_config( - _current_user: AuthContextDep, config_id: UUID + _current_user: AuthContextDep, + config_id: Annotated[UUID, Path(description="Validator config id.")], ) -> Response: status_code, payload = proxy_guardrails_request( "GET", @@ -257,10 +392,19 @@ def get_guardrails_validator_config( @router.patch( "/guardrails/validators/configs/{config_id}", + summary="Update a validator config", + description=load_description("guardrails/update_validator_config.md"), dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], + **doc( + request_schema=ref("GuardrailsValidatorConfigUpdate"), + request=VALIDATOR_CONFIG_UPDATE_EXAMPLE, + responses={200: VALIDATOR_CONFIG_OK, **PROXY_ITEM_WRITE_RESPONSES}, + ), ) def update_guardrails_validator_config( - _current_user: AuthContextDep, config_id: UUID, body: dict[str, Any] + _current_user: AuthContextDep, + config_id: Annotated[UUID, Path(description="Validator config id.")], + body: ProxyBody = None, ) -> Response: status_code, payload = proxy_guardrails_request( "PATCH", @@ -274,10 +418,14 @@ def update_guardrails_validator_config( @router.delete( "/guardrails/validators/configs/{config_id}", + summary="Delete a validator config", + description=load_description("guardrails/delete_validator_config.md"), dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], + **doc(responses={200: DELETE_OK, **PROXY_ITEM_RESPONSES}), ) def delete_guardrails_validator_config( - _current_user: AuthContextDep, config_id: UUID + _current_user: AuthContextDep, + config_id: Annotated[UUID, Path(description="Validator config id.")], ) -> Response: status_code, payload = proxy_guardrails_request( "DELETE", @@ -290,10 +438,14 @@ def delete_guardrails_validator_config( @router.get( "/guardrails/ban_lists/{ban_list_id}", + summary="Get a ban list", + description=load_description("guardrails/get_ban_list.md"), dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], + **doc(responses={200: BAN_LIST_OK, **PROXY_ITEM_RESPONSES}), ) def get_guardrails_ban_list( - _current_user: AuthContextDep, ban_list_id: UUID + _current_user: AuthContextDep, + ban_list_id: Annotated[UUID, Path(description="Ban list id.")], ) -> Response: status_code, payload = proxy_guardrails_request( "GET", @@ -306,10 +458,19 @@ def get_guardrails_ban_list( @router.patch( "/guardrails/ban_lists/{ban_list_id}", + summary="Update a ban list", + description=load_description("guardrails/update_ban_list.md"), dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], + **doc( + request_schema=ref("GuardrailsBanListUpdate"), + request=BAN_LIST_UPDATE_EXAMPLE, + responses={200: BAN_LIST_OK, **PROXY_ITEM_WRITE_RESPONSES}, + ), ) def update_guardrails_ban_list( - _current_user: AuthContextDep, ban_list_id: UUID, body: dict[str, Any] + _current_user: AuthContextDep, + ban_list_id: Annotated[UUID, Path(description="Ban list id.")], + body: ProxyBody = None, ) -> Response: status_code, payload = proxy_guardrails_request( "PATCH", @@ -323,10 +484,14 @@ def update_guardrails_ban_list( @router.delete( "/guardrails/ban_lists/{ban_list_id}", + summary="Delete a ban list", + description=load_description("guardrails/delete_ban_list.md"), dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], + **doc(responses={200: DELETE_OK, **PROXY_ITEM_RESPONSES}), ) def delete_guardrails_ban_list( - _current_user: AuthContextDep, ban_list_id: UUID + _current_user: AuthContextDep, + ban_list_id: Annotated[UUID, Path(description="Ban list id.")], ) -> Response: status_code, payload = proxy_guardrails_request( "DELETE", @@ -339,10 +504,14 @@ def delete_guardrails_ban_list( @router.get( "/guardrails/llm_prompt_configs/{prompt_config_id}", + summary="Get an LLM prompt config", + description=load_description("guardrails/get_llm_prompt_config.md"), dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], + **doc(responses={200: LLM_PROMPT_CONFIG_OK, **PROXY_ITEM_RESPONSES}), ) def get_guardrails_llm_prompt_config( - _current_user: AuthContextDep, prompt_config_id: UUID + _current_user: AuthContextDep, + prompt_config_id: Annotated[UUID, Path(description="LLM prompt config id.")], ) -> Response: status_code, payload = proxy_guardrails_request( "GET", @@ -355,10 +524,19 @@ def get_guardrails_llm_prompt_config( @router.patch( "/guardrails/llm_prompt_configs/{prompt_config_id}", + summary="Update an LLM prompt config", + description=load_description("guardrails/update_llm_prompt_config.md"), dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], + **doc( + request_schema=ref("GuardrailsLLMPromptConfigUpdate"), + request=LLM_PROMPT_CONFIG_UPDATE_EXAMPLE, + responses={200: LLM_PROMPT_CONFIG_OK, **PROXY_ITEM_WRITE_RESPONSES}, + ), ) def update_guardrails_llm_prompt_config( - _current_user: AuthContextDep, prompt_config_id: UUID, body: dict[str, Any] + _current_user: AuthContextDep, + prompt_config_id: Annotated[UUID, Path(description="LLM prompt config id.")], + body: ProxyBody = None, ) -> Response: status_code, payload = proxy_guardrails_request( "PATCH", @@ -372,10 +550,14 @@ def update_guardrails_llm_prompt_config( @router.delete( "/guardrails/llm_prompt_configs/{prompt_config_id}", + summary="Delete an LLM prompt config", + description=load_description("guardrails/delete_llm_prompt_config.md"), dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], + **doc(responses={200: DELETE_OK, **PROXY_ITEM_RESPONSES}), ) def delete_guardrails_llm_prompt_config( - _current_user: AuthContextDep, prompt_config_id: UUID + _current_user: AuthContextDep, + prompt_config_id: Annotated[UUID, Path(description="LLM prompt config id.")], ) -> Response: status_code, payload = proxy_guardrails_request( "DELETE", @@ -388,13 +570,16 @@ def delete_guardrails_llm_prompt_config( @router.get( "/guardrails/{job_id}", + summary="Get guardrails job status", + description=load_description("guardrails/get_guardrails_job.md"), response_model=APIResponse[GuardrailsJobPublic], dependencies=[Depends(require_permission(Permission.REQUIRE_PROJECT))], + **doc(responses=JOB_ITEM_RESPONSES), ) def get_guardrails_job_status( _current_user: AuthContextDep, session: SessionDep, - job_id: UUID, + job_id: Annotated[UUID, Path(description="Job id returned by POST /guardrails.")], ) -> APIResponse[GuardrailsJobPublic]: """Poll for a /guardrails job's status and result. diff --git a/backend/app/models/guardrails/__init__.py b/backend/app/models/guardrails/__init__.py index 423311150..a751cb68d 100644 --- a/backend/app/models/guardrails/__init__.py +++ b/backend/app/models/guardrails/__init__.py @@ -14,7 +14,6 @@ __all__ = [ "GuardrailValidator", - "GuardrailsRequest", "GuardrailsCallbackData", "GuardrailsCallbackResponse", "GuardrailsCallbackUsage", @@ -22,4 +21,5 @@ "GuardrailsJobPublic", "GuardrailsOutput", "GuardrailsOutputContent", + "GuardrailsRequest", ] diff --git a/backend/app/tests/api/routes/test_guardrails.py b/backend/app/tests/api/routes/test_guardrails.py index 7ac271b31..b04a7582a 100644 --- a/backend/app/tests/api/routes/test_guardrails.py +++ b/backend/app/tests/api/routes/test_guardrails.py @@ -16,11 +16,31 @@ VALIDATOR_ID = str(uuid4()) +# The proxy routes validate their bodies against the mirrored guardrails +# schemas, so these have to be complete payloads, not stubs. +BAN_LIST_CREATE_BODY = { + "name": "Safety Banned Terms", + "description": "Terms not allowed for this tenant policy", + "domain": "abuse", + "banned_words": ["slur_a", "slur_b"], +} +LLM_PROMPT_CONFIG_CREATE_BODY = { + "validator_name": "topic_relevance", + "name": "Maternal Health Scope", + "description": "Topic guard for maternal health support bot", + "llm_prompt": "Only answer questions about maternal health.", +} +VALIDATOR_CONFIG_CREATE_BODY = { + "name": "PII Redaction Input", + "type": "pii_remover", + "stage": "input", +} + # (client method, kaapi path, request body, upstream path) — "{id}" is filled # with a freshly generated UUID by the tests that consume this table. PROXY_ROUTES = [ ("GET", "/guardrails", None, "/"), - ("POST", "/guardrails/ban_lists", {"name": "slurs"}, "/ban_lists/"), + ("POST", "/guardrails/ban_lists", BAN_LIST_CREATE_BODY, "/ban_lists/"), ("GET", "/guardrails/ban_lists", None, "/ban_lists/"), ("GET", "/guardrails/ban_lists/{id}", None, "/ban_lists/{id}"), ("PATCH", "/guardrails/ban_lists/{id}", {"name": "renamed"}, "/ban_lists/{id}"), @@ -28,7 +48,7 @@ ( "POST", "/guardrails/llm_prompt_configs", - {"validator_name": "toxicity", "prompt": "be nice"}, + LLM_PROMPT_CONFIG_CREATE_BODY, "/llm_prompt_configs/", ), ("GET", "/guardrails/llm_prompt_configs", None, "/llm_prompt_configs/"), @@ -41,7 +61,7 @@ ( "PATCH", "/guardrails/llm_prompt_configs/{id}", - {"prompt": "be nicer"}, + {"llm_prompt": "be nicer"}, "/llm_prompt_configs/{id}", ), ( @@ -53,7 +73,7 @@ ( "POST", "/guardrails/validators/configs", - {"type": "pii", "stage": "input"}, + VALIDATOR_CONFIG_CREATE_BODY, "/validators/configs/", ), ("GET", "/guardrails/validators/configs", None, "/validators/configs/"), @@ -325,21 +345,18 @@ def test_empty_upstream_body_returns_status_with_no_body( def test_create_echoes_upstream_status_and_body( self, client: TestClient, user_api_key_header: dict[str, str] ) -> None: - created = {"id": str(uuid4()), "validator_name": "toxicity"} + created = {"id": str(uuid4()), "validator_name": "topic_relevance"} # Upstream create routes return FastAPI's default 200. with _mock_upstream(status_code=200, json_body=created) as calls: resp = client.post( "api/v1/guardrails/llm_prompt_configs", - json={"validator_name": "toxicity", "prompt": "be nice"}, + json=LLM_PROMPT_CONFIG_CREATE_BODY, headers=user_api_key_header, ) assert resp.status_code == 200 assert resp.json() == created - assert calls[0]["kwargs"]["json"] == { - "validator_name": "toxicity", - "prompt": "be nice", - } + assert calls[0]["kwargs"]["json"] == LLM_PROMPT_CONFIG_CREATE_BODY def test_connect_error_returns_502( self, client: TestClient, user_api_key_header: dict[str, str] @@ -440,12 +457,12 @@ def test_llm_prompt_config_filters_forwarded( ) -> None: with _mock_upstream(json_body={"data": []}) as calls: client.get( - "api/v1/guardrails/llm_prompt_configs?validator_name=toxicity&limit=50", + "api/v1/guardrails/llm_prompt_configs?validator_name=topic_relevance&limit=50", headers=user_api_key_header, ) assert calls[0]["kwargs"]["params"] == { - "validator_name": "toxicity", + "validator_name": "topic_relevance", "offset": 0, "limit": 50, } @@ -455,11 +472,14 @@ def test_validator_config_filters_forwarded_without_ids( ) -> None: with _mock_upstream(json_body={"data": []}) as calls: client.get( - "api/v1/guardrails/validators/configs?stage=output&type=pii", + "api/v1/guardrails/validators/configs?stage=output&type=pii_remover", headers=user_api_key_header, ) - assert calls[0]["kwargs"]["params"] == {"stage": "output", "type": "pii"} + assert calls[0]["kwargs"]["params"] == { + "stage": "output", + "type": "pii_remover", + } def test_ids_are_normalised_to_canonical_uuid_strings( self, client: TestClient, user_api_key_header: dict[str, str] @@ -497,7 +517,11 @@ def test_tenant_headers_come_from_auth_context_not_request( with _mock_upstream(status_code=200, json_body={"id": str(uuid4())}) as calls: client.post( "api/v1/guardrails/ban_lists?organization_id=999", - json={"name": "slurs", "organization_id": 999, "project_id": 888}, + json={ + **BAN_LIST_CREATE_BODY, + "organization_id": 999, + "project_id": 888, + }, headers=user_api_key_header, ) @@ -653,3 +677,61 @@ def _seed_guardrails_job( job_id=job.id, job_update=JobUpdate(status=status, error_message=error_message), ) + + +class TestOpenAPIDocumentation: + """Every guardrails operation must stay testable from the Swagger UI.""" + + @staticmethod + def _guardrails_operations() -> list[tuple[str, str, dict[str, Any]]]: + from app.main import app + + schema = app.openapi() + return [ + (path, method, operation) + for path, item in schema["paths"].items() + if path.startswith("/api/v1/guardrails") + for method, operation in item.items() + if method in ("get", "post", "patch", "delete") + ] + + def test_every_operation_is_documented(self) -> None: + for path, method, operation in self._guardrails_operations(): + assert operation.get("summary"), f"{method.upper()} {path} has no summary" + assert operation.get( + "description" + ), f"{method.upper()} {path} has no description" + + def test_every_operation_declares_a_response_schema(self) -> None: + for path, method, operation in self._guardrails_operations(): + schema = ( + operation["responses"]["200"] + .get("content", {}) + .get("application/json", {}) + .get("schema", {}) + ) + assert schema, f"{method.upper()} {path} documents no 200 schema" + + def test_write_operations_declare_a_typed_request_body(self) -> None: + """An untyped body renders an empty box in Swagger, which is untestable.""" + for path, method, operation in self._guardrails_operations(): + if method not in ("post", "patch"): + continue + schema = operation["requestBody"]["content"]["application/json"]["schema"] + assert schema.get("$ref"), f"{method.upper()} {path} has an untyped body" + + def test_apply_guardrails_publishes_its_webhook_contract(self) -> None: + from app.main import app + + operation = app.openapi()["paths"]["/api/v1/guardrails"]["post"] + callback = operation["callbacks"]["guardrails_callback"]["{$callback_url}"] + schema = callback["post"]["requestBody"]["content"]["application/json"][ + "schema" + ] + assert schema.get("$ref"), "webhook payload contract is not published" + + def test_guardrails_tag_is_described_and_grouped(self) -> None: + from app.api.docs.openapi_config import tag_groups, tags_metadata + + assert any(tag["name"] == "Guardrails" for tag in tags_metadata) + assert any("Guardrails" in group["tags"] for group in tag_groups) diff --git a/backend/app/tests/api/test_openapi_docs.py b/backend/app/tests/api/test_openapi_docs.py new file mode 100644 index 000000000..599c31fd0 --- /dev/null +++ b/backend/app/tests/api/test_openapi_docs.py @@ -0,0 +1,201 @@ +from typing import Any + +import pytest + +from app.api import openapi_docs +from app.api.openapi_docs import ( + JSON_MEDIA_TYPE, + ResponseDoc, + doc, + infer_schema, + inject_component_schemas, + ref, + register_component_schemas, +) + + +@pytest.fixture +def clean_registry(): + """Isolate the module-level component registry. + + It is global and populated at import by `openapi_config`, so a test that + registered into it would otherwise leak schemas into every later test. + """ + original = dict(openapi_docs._REGISTERED_COMPONENTS) + openapi_docs._REGISTERED_COMPONENTS.clear() + yield openapi_docs._REGISTERED_COMPONENTS + openapi_docs._REGISTERED_COMPONENTS.clear() + openapi_docs._REGISTERED_COMPONENTS.update(original) + + +def _body(kwargs: dict[str, Any]) -> dict[str, Any]: + """The JSON content block of a `doc()` request body.""" + return kwargs["openapi_extra"]["requestBody"]["content"][JSON_MEDIA_TYPE] + + +class TestInferSchema: + """Schemas inferred from example values.""" + + @pytest.mark.parametrize( + "value, expected", + [ + (True, {"type": "boolean"}), + (3, {"type": "integer"}), + (1.5, {"type": "number"}), + ("text", {"type": "string"}), + (None, {"nullable": True}), + ], + ) + def test_scalars(self, value: Any, expected: dict[str, Any]) -> None: + assert infer_schema(value) == expected + + def test_bool_is_not_reported_as_integer(self) -> None: + """`bool` subclasses `int`, so the order of the checks is load-bearing.""" + assert infer_schema(False) == {"type": "boolean"} + + def test_list_takes_its_item_schema_from_the_first_element(self) -> None: + assert infer_schema(["a", "b"]) == { + "type": "array", + "items": {"type": "string"}, + } + + def test_empty_list_leaves_items_open(self) -> None: + assert infer_schema([]) == {"type": "array", "items": {}} + + def test_dict_recurses_into_properties(self) -> None: + assert infer_schema({"name": "pen", "tags": [1]}) == { + "type": "object", + "properties": { + "name": {"type": "string"}, + "tags": {"type": "array", "items": {"type": "integer"}}, + }, + } + + def test_unsupported_value_yields_an_empty_schema(self) -> None: + """Better an unconstrained schema than a spec that fails to load.""" + assert infer_schema({"a", "b"}) == {} + + +class TestRef: + def test_points_at_the_components_section(self) -> None: + assert ref("BanList") == {"$ref": "#/components/schemas/BanList"} + + +class TestComponentRegistry: + def test_registered_schemas_are_injected(self, clean_registry) -> None: + register_component_schemas({"Thing": {"type": "object"}}) + spec: dict[str, Any] = {} + + injected = inject_component_schemas(spec) + + assert injected["components"]["schemas"]["Thing"] == {"type": "object"} + + def test_injection_keeps_existing_components(self, clean_registry) -> None: + register_component_schemas({"Thing": {"type": "object"}}) + spec = {"components": {"schemas": {"Other": {"type": "string"}}}} + + schemas = inject_component_schemas(spec)["components"]["schemas"] + + assert sorted(schemas) == ["Other", "Thing"] + + def test_generated_component_wins_on_a_name_clash(self, clean_registry) -> None: + """A real Pydantic model always describes a route better than a stand-in.""" + register_component_schemas({"Thing": {"type": "object"}}) + spec = {"components": {"schemas": {"Thing": {"title": "generated"}}}} + + schemas = inject_component_schemas(spec)["components"]["schemas"] + + assert schemas["Thing"] == {"title": "generated"} + + def test_injection_mutates_the_spec_in_place(self, clean_registry) -> None: + register_component_schemas({"Thing": {"type": "object"}}) + spec: dict[str, Any] = {} + + assert inject_component_schemas(spec) is spec + + +class TestDoc: + def test_no_arguments_documents_nothing(self) -> None: + assert doc() == {} + + def test_request_example_is_documented_with_an_inferred_schema(self) -> None: + kwargs = doc(request={"name": "pen"}) + + assert kwargs["openapi_extra"]["requestBody"]["required"] is True + assert _body(kwargs) == { + "schema": {"type": "object", "properties": {"name": {"type": "string"}}}, + "example": {"name": "pen"}, + } + + def test_explicit_schema_overrides_inference(self) -> None: + kwargs = doc(request={"name": "pen"}, request_schema=ref("BanList")) + + assert _body(kwargs)["schema"] == ref("BanList") + + def test_schema_alone_documents_a_body_without_an_example(self) -> None: + kwargs = doc(request_schema=ref("BanList")) + + assert _body(kwargs) == {"schema": ref("BanList")} + + def test_examples_render_as_a_named_map(self) -> None: + kwargs = doc( + request_schema=ref("BanList"), + request_examples={"minimal": ("Smallest body", {"name": "pen"})}, + ) + + assert _body(kwargs)["examples"] == { + "minimal": {"summary": "Smallest body", "value": {"name": "pen"}} + } + assert "example" not in _body(kwargs) + + def test_examples_without_a_schema_infer_from_the_first_one(self) -> None: + """Inferring from the absent `request` would document a nullable body.""" + kwargs = doc(request_examples={"minimal": ("Smallest body", {"name": "pen"})}) + + assert _body(kwargs)["schema"] == { + "type": "object", + "properties": {"name": {"type": "string"}}, + } + + def test_body_can_be_marked_optional(self) -> None: + kwargs = doc(request={"name": "pen"}, required=False) + + assert kwargs["openapi_extra"]["requestBody"]["required"] is False + + def test_response_schema_is_inferred_from_its_example(self) -> None: + kwargs = doc(responses={404: ResponseDoc("Not found", {"error": "gone"})}) + + assert kwargs["responses"][404] == { + "description": "Not found", + "content": { + JSON_MEDIA_TYPE: { + "schema": { + "type": "object", + "properties": {"error": {"type": "string"}}, + }, + "example": {"error": "gone"}, + } + }, + } + + def test_explicit_response_schema_is_kept(self) -> None: + kwargs = doc( + responses={ + 200: ResponseDoc("The ban list", {"id": 1}, ref("BanListResponse")) + } + ) + + content = kwargs["responses"][200]["content"][JSON_MEDIA_TYPE] + assert content["schema"] == ref("BanListResponse") + assert content["example"] == {"id": 1} + + def test_request_and_responses_are_documented_together(self) -> None: + kwargs = doc( + request={"name": "pen"}, + responses={401: ResponseDoc("Unauthorized", {"error": "nope"})}, + ) + + assert sorted(kwargs) == ["openapi_extra", "responses"] + + def test_empty_responses_adds_no_key(self) -> None: + assert doc(responses={}) == {} diff --git a/docs/wiki/modules/guardrails.md b/docs/wiki/modules/guardrails.md index 0efa0806a..ca284d544 100644 --- a/docs/wiki/modules/guardrails.md +++ b/docs/wiki/modules/guardrails.md @@ -21,7 +21,8 @@ Both funnel into `apply_guardrails()` in `services/llm/guardrails.py`. That modu | Standalone job orchestration (dedupe, warnings, callbacks) | `services/guardrails/jobs.py` | | Routes: async job + poll + management proxies | `api/routes/guardrails.py` | | Public endpoint contract (rendered into OpenAPI) | `api/docs/guardrails/apply_guardrails.md` | -| Request/response schemas | `models/guardrails/request.py`, `models/guardrails/response.py` | +| Request/response schemas for the **job** routes (validated) | `models/guardrails/request.py`, `models/guardrails/response.py` | +| Documented-but-not-validated contract for the **proxy** routes | `api/docs/guardrails_schemas.py` (JSON Schemas + examples), `api/openapi_docs.py` (`doc()`/`ref()` helpers, component registry) | | Inline adapters + call sites | `services/llm/jobs.py`, `services/llm/chain/chain.py` | | Celery task + enqueue helper | `celery/tasks/job_execution.py` (`run_guardrails_job`), `celery/utils.py` (`start_guardrails_job`) | | `JobType.LLM_GUARDRAILS`, `job.meta` | `models/job.py` | @@ -110,13 +111,25 @@ Thin pass-throughs over the upstream management API, all under `require_permissi These do **not** fail open: upstream status codes and bodies (including its 422s) are returned unchanged. `_upstream_response` sets `Cache-Control: no-store` (tenant-scoped data must not sit in a shared cache, CWE-525) and keeps an empty upstream body empty, since a 204 cannot carry one. A non-JSON upstream body becomes a 502. +**Bodies are not validated here.** Each write route takes a `ProxyBody` (`Annotated[Any, Body(...)]`) and forwards it verbatim; the guardrails service owns validation. Mirroring its schemas as Pydantic models was tried and abandoned: the mirror rejected payloads upstream accepts as soon as the two drifted — a `content_safety` validator type added upstream was a gateway 422 while the enum here lagged behind. + +`GET /guardrails` carries a third thing: `VALIDATOR_TYPES` in `api/docs/guardrails_schemas.py`, a catalogue of all 12 validator types with their tuning keys, defaults and gotchas. It renders *both* the example response and the per-type reference appended to the route description (`render_validator_reference()`), so those two cannot disagree with each other. They can still fall behind the guardrails service — that is the one piece of this module that needs re-checking when upstream adds a validator or a key. Re-verify by diffing against the live schemas: + +```python +# in kaapi-guardrails +from app.core.guardrail_controller import get_validator_config_models +[m.model_json_schema() for m in get_validator_config_models()] +``` + +Swagger stays fully typed anyway. `api/docs/guardrails_schemas.py` holds hand-written JSON Schemas and examples; `register_component_schemas()` publishes them under `components.schemas` (injected by `customize_openapi_schema`), and each route `$ref`s them through `doc(request_schema=ref(...), responses={...})`. Docs metadata only — it cannot reject a request. Query params follow the same rule: filters are plain `str`, so an unknown-but-valid-upstream value is forwarded rather than 422'd. Pagination `ge`/`le` bounds are the one deliberate exception, since an out-of-range page is a client bug either way. + ## 9. Invariants — do not break these 1. Tenant identity travels in headers derived from the auth context. Never accept `organization_id`/`project_id` from a request body or query string. 2. 401/403/422 fail closed; network/5xx fail open. Changing either side changes the security posture — say so explicitly in the PR. 3. Client-visible error strings stay status-only, never `str(e)`. 4. Fixed `/guardrails/*` paths stay declared **above** `GET /guardrails/{job_id}`. FastAPI matches in declaration order and will not fall through on a UUID parse failure. -5. Management proxies return upstream status/body verbatim with `no-store`. +5. Management proxies return upstream status/body verbatim with `no-store`, and validate **nothing** locally. A new field or validator type upstream must reach it without a change here; document it in `api/docs/guardrails_schemas.py` instead of enforcing it. 6. Metadata emission stays gated on `outcome.applied`. 7. Raw text persistence is intentional on the `/guardrails` job, not an oversight. 8. `guardrail_outcome` marks content verdicts only. Never label a fail-closed auth/transport error `"blocked"` — a caller that treats blocks as benign (evaluation does) would silently swallow a broken deploy. It must not be retried either: the credentials are as broken on the third attempt as on the first, and an *output*-side failure re-charges the provider each time, because the completion is generated before output guardrails run. Hence `retryable` stays `False` on both guardrail error paths. @@ -128,7 +141,8 @@ These do **not** fail open: upstream status codes and bodies (including its 422s | Add/modify an output guardrail behaviour | `apply_output_guardrails`, then `_run_output_guardrails` in `jobs.py` (shared by the proxy and provider paths) | | Change `apply_input_guardrails`' return | Add a field to `GuardrailResult` in `jobs.py`, then read it in `_run_input_guardrails` (`jobs.py`) | | Add a field to the `/guardrails` result | `models/guardrails/response.py` → `_build_callback_payload` (callback path) → `get_guardrails_job_status` (poll path, rebuilds independently) → `api/docs/guardrails/apply_guardrails.md` | -| Add a management proxy route | `api/routes/guardrails.py` using `proxy_guardrails_request` + `_upstream_response`, declared above `/{job_id}` | +| Add a management proxy route | `api/routes/guardrails.py` using `proxy_guardrails_request` + `_upstream_response`, declared above `/{job_id}`; body typed `ProxyBody`, docs via `**doc(...)` with schemas/examples added to `api/docs/guardrails_schemas.py` | +| Upstream adds a field or validator type | Nothing is required for it to work — it already forwards. Update `VALIDATOR_TYPES` (and the relevant schema/example) in `api/docs/guardrails_schemas.py` so the docs stay honest | | Change upstream request/response shape | `run_guardrails_validation` and/or `list_validators_config`; check `summarize_validator_results` still finds `data.validator_results` | | Add a new warning | `_dedupe_validators` or `_outcome_warnings` in `services/guardrails/jobs.py`; document it in the api doc | | Change timeouts | `services/llm/guardrails.py` (three separate values, see §2) | @@ -159,7 +173,7 @@ GET /validators/configs/?ids=... -> {success, data: [ {...validator config...} | File | Covers | |---|---| -| `tests/api/routes/test_guardrails.py` | Routes, proxy behaviour, route ordering, poll | +| `tests/api/routes/test_guardrails.py` | Routes, proxy behaviour, route ordering, poll, and `TestOpenAPIDocumentation` (every operation carries a summary, a 200 schema and a `$ref`'d request body) | | `tests/services/llm/test_guardrails.py` | Transport, fail-open/fail-closed matrix | | `tests/services/guardrails/test_jobs.py` | Standalone job worker, dedupe, warnings, callbacks | | `tests/services/llm/test_jobs.py` | Inline adapters, metadata emission |