Meta-harness optimization loop wired onto Islo sandboxes. POC: 0/5→5/5 in four proposer steps. Built on islo.dev.
-
Updated
May 5, 2026 - HTML
#
agent-eval
Here are 30 public repositories matching this topic...
Lynn: open-source desktop/CLI AI Agent with GUI Session Map, headless workers, realtime voice, long-term memory, Brain V2 routing, local 27B/35B GGUF support, and contract-based Agent Regression Kit.
electron benchmark typescript regression-testing huggingface ai-agent realtime-voice modelscope desktop-agent openai-compatible cli-agent agent-runtime coding-benchmark agent-eval agent-regression
-
Updated
Jul 13, 2026 - TypeScript
Observe an agent run and GroundEval drafts the policy and diagram for you, no hand-written policy required, then scores what it checked, what it skipped, and what it wasn't allowed to touch.
benchmarking evaluation-framework ai-agents test-agents llm-evaluation llm-as-judge llm-as-a-judge agent-evals agent-evaluation agent-harness agent-loop agent-eval debug-agents
-
Updated
Jul 17, 2026 - Python
Live, open-source benchmark for comparing AI coding agents on real GitHub issues
benchmark machine-learning developer-tools awesome-list dev-tools live-data ai-research ai-benchmarks ai-engineering ai-tools auto-updated ai-evaluation llm-testing agent-evaluation swe-bench llm-benchmarks agent-eval ai-coding-agent-benchmark codex-vs-opencode coding-agent-benchmark
-
Updated
Jul 26, 2026 - Python
Framework-agnostic evaluation harness for Go — test your MCP servers and AI agents with scored, CI-ready checks.
-
Updated
Jun 17, 2026 - Go
AI-operated company. Building agent-friend: universal tool adapter for AI agents. @tool → OpenAI, Claude, Gemini, MCP. Live 24/7 on Twitch.
python twitch structured-logging interactive-cli exponential-backoff human-in-the-loop zero-dependencies open-startup ai-agent autonomous-ai building-in-public llm-tools agent-security mcp-security personal-ai-agent agent-eval agent-friend
-
Updated
Jul 5, 2026 - Python
Project page for Meta-harness on Islo (POC). https://zozo123.github.io/meta-harness-on-islo-page/
-
Updated
May 5, 2026 - JavaScript
A reasoning benchmark runner for comparing LLMs as OpenClaw agents use them. 52 prompts, 3 eval sets, 11 traps, LLM-as-judge, tier-based leaderboard.
-
Updated
Apr 11, 2026 - Python
Scenario Testing for AI Agents
-
Updated
Jul 25, 2026 - Python
Vendor-neutral research umbrella for measuring AI plugin, agent, and MCP server quality across CLI runtimes (Claude Code, Gemini CLI, Copilot CLI, Codex CLI).
mcp skill-discovery opentelemetry ai-evaluation gemini-cli claude-code plugin-testing cross-cli agent-eval invocation-rate
-
Updated
Jul 26, 2026 - Python
Transcript-first evaluation tool for comparing coding-agent sessions across Codex, Claude Code, and Pi.
-
Updated
May 29, 2026 - TypeScript
Deterministic execution recording and replay for AI agents — turn real agent runs into replayable regression test fixtures.
python pytest ai-agents drift-detection record-replay crewai langgraph agent-testing deterministic-execution agent-eval replay-testing openeval execution-assurance agent-fixtures
-
Updated
Jul 26, 2026 - Python
开源通用 AI Agent 真实任务评测 · 同 Prompt、客观开奖、评分细则全公开 | Open-source evaluation of general-purpose AI Agents on real-world tasks with verifiable outcomes — by PingWest / 硅星人
-
Updated
Jun 13, 2026
ckl's personal benchmark for doc writing, infra code, and paper reading — one-click evaluation of the latest models via TUI coding CLIs with an interactive dashboard.
-
Updated
Jul 22, 2026 - HTML
Trajectory-level evaluation for LLM agents — catch cross-turn defects that naive per-turn LLM-as-judge misses, with honest recall measurement
-
Updated
Jul 1, 2026 - TypeScript
96K param RWKV-7 that detects non-termination (the Gödel sentence analog) zero-shot across SKI combinatory logic, lambda calculus, and Turing machines
-
Updated
Jun 15, 2026 - Python
Auto-generate evaluation rubrics from agent audit-log trajectories (PhoneWorld pattern applied to action logs)
-
Updated
Jun 23, 2026 - Python
🚀 基于Java的开源AI自动化评测框架 / An open source AI automation evaluation framework based on Java
-
Updated
Jun 30, 2026 - Java
An AI coding benchmark task evaluating automated rebase conflict resolution, MLflow tracking security, secret redaction, and Hugging Face model evaluation safety.
python3 git-rebase cybersecurity security-hardening mlops mlflow huggingface agent-evaluation agent-eval benchmark-tasks
-
Updated
Jul 25, 2026 - Python
PandaProbe Harness turns your agent into a self-healing harness
-
Updated
Jul 24, 2026 - Python
Improve this page
Add a description, image, and links to the agent-eval topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the agent-eval topic, visit your repo's landing page and select "manage topics."