Build Your Own Agent Red-Team Evaluator

A CodeCrafters-style, public-safe challenge track for building an AI agent red-team evaluation harness from scratch.

You will build a small evaluator that can model risks, apply attacks, run an agent under test, judge the response, generate findings, and escalate ambiguous cases to manual review.

Risk + Attack + Prompt
        |
        v
Attack Transform
        |
        v
Agent Under Test
        |
        v
Judge
        |
        v
Finding + Eval Report

This repo is sanitized for public sharing. It contains no private recruiter messages, calendar links, resumes, email bodies, API keys, or private interview artifacts.

The Challenge

Build your own AI agent red-team harness in stages:

Stage	You Build	Why It Matters
1	Risk, Attack, TestCase, Finding	Turns messy safety concerns into typed eval objects
2	Attack transforms	Separates attack technique from the risk being tested
3	Red-team runner	Creates the execution lifecycle
4	Deterministic judges	Handles cheap, stable checks like leakage and forbidden actions
5	Manual eval queue	Captures cases humans should review
6	LLM-as-judge adapter	Handles semantic judgment where regex is too brittle
7	SDK adapters	Shows how OpenAI, Claude, and pi-ai style routers fit in
8	Report + remediation loop	Produces evidence-backed outputs a team can act on

Quickstart

git clone https://github.com/HomenShum/AgentRedteam
cd AgentRedteam
python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -e .[dev]
python -m pytest -q
python -m redteam_eval_lab.cli --judge deterministic
python -m redteam_eval_lab.cli --judge manual

The tests should pass. The CLI intentionally reports failures because the toy agent is vulnerable.

Optional real LLM judges:

pip install -e .[openai]
$env:OPENAI_API_KEY="..."
python -m redteam_eval_lab.cli --judge openai
python -m redteam_eval_lab.cli --judge openai-chat
python -m redteam_eval_lab.cli --judge openai-agents

pip install -e .[anthropic]
$env:ANTHROPIC_API_KEY="..."
python -m redteam_eval_lab.cli --judge anthropic

Start Here

If you want the CodeCrafters-style path, read the stages in order:

1. Stage 1 - Define the eval schema
2. Stage 2 - Add attack transforms
3. Stage 3 - Build the runner
4. Stage 4 - Add deterministic judges
5. Stage 5 - Add manual eval
6. Stage 6 - Add LLM-as-judge
7. Stage 7 - Add SDK adapters
8. Stage 8 - Ship reports and remediation

Use the starter kit:

• Python starter
• Starter tests
• Completed solution
• Completed code examples
• Grader
• Reference implementation

Run a staged grader:

python challenge/grader.py --stage 01 --impl starter
python challenge/grader.py --stage 08 --impl solution

Reference Implementation

The working implementation lives in src/redteam_eval_lab.

Important files:

• schemas.py - risk, attack, testcase, finding, report
• attacks.py - prompt injection, base64, JSON injection, hidden markdown
• agents.py - intentionally vulnerable toy agent
• judges.py - deterministic, manual, OpenAI, Anthropic judges
• llm_clients.py - real OpenAI, Anthropic, and OpenAI Agents JSON clients
• runner.py - orchestration loop
• suites.py - sample risk/attack test cases

AgentUnderTest is a protocol/interface, so its respond() method is only a contract. Concrete implementations include ToyAgent and EchoAgent; real apps would provide an adapter around an SDK, local service, or deployed agent.

Judge Design

Production systems rarely use just one judge:

Deterministic checks
  -> schema validation
  -> LLM judge
  -> second judge for disputed cases
  -> manual review
  -> remediation tracking

Judge	Use For	Strength	Weakness
Deterministic	Terms, schemas, tool-call permissions	Fast and stable	Misses nuance
LLM judge	Hallucination, grounding, policy adherence	Handles semantics	Costs money and can drift
Manual eval	Ambiguous or high-stakes findings	Best calibration source	Slow
Hybrid	Real production loops	Balanced	More system complexity

SDK Adapter Examples

The repo includes optional patterns for:

• OpenAI Agents SDK / OpenAI API
• Claude Agent SDK / Anthropic API
• pi-ai style model routing
• manual eval queues

The default test suite does not need API keys.

See:

• Judge Design
• SDK Adapters
• Manual Eval Workflow
• Interview Prep Notes

Interview Soundbite

I would start with deterministic checks for cheap, stable failures, then use LLM-as-judge for semantic cases like hallucination, policy adherence, grounding, and tone. For ambiguous or high-impact findings, I would route to manual review. The key is that all judges return the same structured Finding, so the rest of the eval pipeline stays stable.

Public-Safety Boundary

This repo teaches the evaluation architecture without publishing private context. Keep real resumes, emails, interview prompts, meeting links, and private API keys out of the repo.