Update 07/2026: The benchmark landscape has shifted fundamentally. Classic benchmarks (MMLU, HumanEval, GSM8K) are saturated - frontier models score >90% and no longer differentiate. For assessing SOTA models in 2026, look at three things instead: composite indexes for the overview, agentic/long-horizon evals for differentiation, and autonomy measurements for the trend.
Composite indexes: the SOTA overview
Artificial Analysis has become the most cited independent reference. Their Intelligence Index (v4.1) is a weighted average over 9 evaluations in 4 categories - Agents 34% (GDPval-AA, τ³-Banking), Coding 24% (Terminal-Bench, SciCode), Scientific Reasoning 24% (HLE, GPQA Diamond, CritPt), General 18% (incl. a hallucination-aware knowledge eval). They also publish a Coding Agent Index, an Openness Index for open-weight models, and consistently measured cost/speed/verbosity data.
Know the limits of composite indexes: the weighting is a judgment call, index version changes break time series, and a single number hides domain weaknesses. Use the index for shortlisting, then check the category scores that match your use case.
Autonomy: METR time horizons
METR measures the task-completion time horizon: how long (in human-expert working time) can a task be such that the model still succeeds with 50% probability. This is the only established metric that expresses progress as autonomy duration rather than benchmark percent - directly relevant for judging what agentic workloads a model can carry. The measured horizon currently doubles roughly every ~4 months; estimates beyond ~16h are considered unreliable, and the suite covers mainly coding/research tasks. Raw data is public.
Coding & agentic evals
- SWE-bench Verified - real GitHub issues, the de-facto coding standard; increasingly saturated at the top and with known contamination concerns.
- DeepSWE (Datacurve, since May 2026) - contamination-free by design: 113 freshly written long-horizon tasks across 5 languages (average solution ~670 lines across 7 files), behavior-testing verifiers. Methodically convincing and already used by Artificial Analysis, but young and run by a commercial data vendor - watch how it establishes itself.
- Terminal-Bench and OSWorld - agentic terminal/computer use.
- BFCL V4 - function calling with agentic evaluation.
- GDPval - economically relevant knowledge work.
- ARC-AGI-2 - abstraction/reasoning differentiation at the frontier.
Still recommended
LMArena (user-preference elo - measures preference, not correctness), MTEB for embeddings, Vectara Hallucination Leaderboard (relaunched May 2026 with a harder dataset). Removed: the Hugging Face Open LLM Leaderboard was officially retired in June 2025.
General caution: vendor-reported scores at model launch are marketing until independently reproduced - prefer leaderboards with independent methodology and published raw data.
Model benchmarks and leaderboards are available for various evaluation dimensions relevant to model choosing and discovery.
In this article we will provide an overview of the most relevant benchmarks and leaderboards, as well as a summary of model limitations that one should have in mind.
Performance & Benchmarks (MMLU etc.):
As large language models (LLMs) and generative AI (GenAI) are rapidly advancing, robust and targeted benchmarks are essential to assess their real-world capabilities, domain-specific expertise, reasoning power, thrustworthiness and environmental impact. The following benchmarks represent a selection of most relevant benchmarks used to evaluate the next generation of AI models.
Overview Important Benchmarks:
| Category | Benchmark(s) | Purpose & Focus Area |
|---|---|---|
| General Knowledge & Reasoning | MMLU-Pro, GPQA, BIG-Bench Hard, AGIEval, Humanity’s Last Exam | Measures domain-specific and general expertise (law, physics, medicine) at different difficulty. |
| Mathematical Reasoning | Math (OpenAI), MathVista, FrontierMath | Symbolic and numerical reasoning, including chain-of-thought and visual math. Mathematic task solving. |
| Instruction Following & Multi-Hop Logic | IFEval, MUSR, LongBench | Evaluates complex instruction following, task planning, and stepwise logic |
| Data Analytics & Querying | BIRD, DataSciBench, Spider 2.0 | SQL generation, structured data understanding, table-to-text generation |
| Coding, Software & Data Science | LiveCodeBench, DS-1000, CodeContests, MultiPL-E, DSBench | Functional correctness of code generation, Complex algorithmic tasks, data science workflows (e.g. pandas, sklearn), multi-programming language, data science expert tasks |
| Multimodal Reasoning | MMMU, MathVista | Text + image reasoning in STEM and professional domains |
| Tool Use & Agent Behavior | ToolBench, WebArena | Real-world agent performance using tools, APIs, browsers |
| Specific Tool Benchmarks | SpreadSheetBench,BrowseComp | Benchmarks specific for common tools like Browser, Excel, .. |
| Safety & Trustworthiness | TruthfulQA, ToxiGen, HHEM | Hallucination resistance, ethical alignment, toxicity filtering |
| Efficiency & Environmental Impact | CO2-Cost | Energy usage and carbon emissions of training/inference pipelines |
| Vision Models (LVLMs) & Multimodality | VLMEvalKit, MMMU, VizWiz | Evaluation of Models that can deal with Image and Text |
Leaderboards
1.Popular Leaderboards
- Chatbot Arena LLM Leaderboard: Community-driven Evaluation for Best LLM and AI chatbots
- Artificial Analysis Leaderboards
- Hugging Face List of Leaderboards on the Hub & Model Catalogs in Hugging Face
- Multilingual MMLU
2. Open Model Benchmarks:
3. Multimodal & Vision Model:
- OpenVLM Leaderboard of VLMEvalKit Evaluations
- MMMU Leaderboard - Understanding and Reasoning Benchmark for Expert AGI
- TTS (Text to Speech) natural sounding Arena
4. Function-Calling:
5. Code:
6. Embedding Models:
7. Safety & Trustworthiness / Censored Models:
8. Cost (Model Provider Cost):
9. Hardware Performance:
9. Hallucination:
Links
- For a broader overview of model discovery, see Model Discovery.