Robust evals for agents

Open-source eval runtime for agents. Measure accuracy, tokens, duration, and capture full execution traces.

Daily benchmarks powered by Margin Eval, with statistical degradation detection.

MARGIN EVALS

Open source orchestrator for robust, reproducible agent evals.

curl -fsSL https://raw.githubusercontent.com/Margin-Lab/evals/main/scripts/install.sh | bash

margin run \
  --suite git::https://github.com/Margin-Lab/swe-suites.git//swe-bench-pro \
  --agent-config example-agent-configs/claude-code-default \
  --eval example-eval-configs/default.toml

Support for arbitrary agents

Claude Code

Codex

OpenCode

Gemini CLI

Warp Code

Cursor

Support for arbitrary benchmarks

0 + built in evals

Read the docs →

TRACKERS

Daily benchmarks powered by Margin Eval, with statistical degradation detection.

Claude Code

Nominal

Baseline

56%

7-day

55%

30-day

57%

Degradation Tracker → Historical →