Assess
Inspect is the open-source (MIT) evaluation framework of the UK AI Security Institute. Evals are defined as reproducible Python tasks (dataset + solver + scorer), with first-class support for agent evaluation: sandboxed tool use, multi-agent setups and a local results UI.
The companion package inspect_evals ships 200+ ready benchmarks - including agentic ones (GAIA, SWE-bench, Cybench) and security suites aligned with the OWASP Agentic Top 10.
Methodically the most rigorous open-source option for agent evals, and no commercial platform behind it - it is publicly funded infrastructure.
Assess: evaluate it when serious agent evaluation is on the roadmap.