Which ways of building AI agents actually work?
HyperAgility lines up the results from 571 research papers on building LLM agents, so you can see which techniques beat the alternatives, on which tasks and models, what they cost, and how they fail. Data as of 2026-11-09. Read these results as Markdown.
How to read these results
- Won means a method scored better than whatever that paper compared it against, on the same model, task and metric. Each paper counts once. Results within half a percent count as tied.
- Typical change is the median relative change in score across papers.
- Independent tests exclude papers introducing that method. Most published results come from papers testing their own new technique, so compare both columns.
- Many results rest on few papers. Treat anything under 5 papers as a lead, and check the source paper before relying on a specific result.
Methods, best supported first
| Method | Won (all results) | Won (independent tests) | Won (same budget) | Typical change |
|---|---|---|---|---|
| Harness search & evolution | 18 of 20 | 5 of 6 | 1 of 2 | +14% |
| Training data & environment synthesis | 16 of 20 | 6 of 7 | none | +14% |
| World-model simulation & lookahead | 8 of 9 | 1 of 3 | none | +15% |
| Reinforcement learning (verifiable / judged rewards) | 65 of 99 | 21 of 31 | 3 of 7 | +8% |
| Skill / playbook self-evolution | 19 of 26 | 10 of 13 | 5 of 6 | +6% |
| Parallel sampling & selection (Best-of-N, voting) | 47 of 72 | 33 of 43 | 8 of 15 | +2% |
| Tree search over actions or solutions | 22 of 32 | 15 of 18 | 2 of 5 | +5% |
| Trained reward / process reward models | 12 of 16 | 6 of 6 | 2 of 3 | +5% |
| Trained memory / context managers | 12 of 16 | 6 of 9 | 2 of 2 | +23% |
| Automated agentic workflow / MAS search | 18 of 26 | 12 of 15 | 2 of 2 | +6% |
| Skills, playbooks & rule files | 28 of 44 | 14 of 20 | 7 of 10 | +5% |
| Experience & lesson memory | 25 of 42 | 18 of 28 | 3 of 3 | +4% |
| Runtime safety guardrails | 5 of 6 | 2 of 3 | none | +29% |
| Hand-written instructions, policies & personas | 31 of 55 | 18 of 33 | 18 of 25 | +3% |
| Supervised fine-tuning on agent trajectories | 64 of 124 | 30 of 49 | 5 of 9 | +2% |
| Context compaction & history management | 23 of 40 | 12 of 20 | 2 of 4 | +2% |
| Structured indexes & knowledge navigation | 10 of 15 | 1 of 4 | 4 of 4 | +19% |
| Role-specialised multi-agent pipeline | 40 of 76 | 14 of 27 | 5 of 9 | +2% |
| Runtime control policies (gates, reminders, budgets) | 18 of 31 | 4 of 10 | 4 of 7 | +4% |
| Automated prompt optimisation | 11 of 18 | 7 of 9 | 0 of 1 | +3% |
| Communication topology & protocol design | 13 of 23 | 6 of 12 | 3 of 6 | +3% |
| Action space & tool-surface design | 22 of 43 | 14 of 22 | 10 of 13 | +1% |
| Standard ReAct-style tool loop | 42 of 97 | 29 of 53 | 7 of 20 | -2% |
| Heterogeneous model composition & routing | 19 of 39 | 8 of 14 | 0 of 1 | +0% |
| Self-reflection & iterative self-refinement | 27 of 59 | 20 of 35 | 4 of 11 | +0% |
| External critic / verifier feedback | 18 of 37 | 11 of 19 | 2 of 6 | +0% |
| Orchestrator with dynamic sub-agents | 17 of 35 | 12 of 22 | 2 of 6 | +0% |
| Chain-of-thought & reasoning elicitation | 30 of 71 | 22 of 44 | 4 of 9 | +0% |
| Plan-then-execute & task decomposition | 21 of 47 | 9 of 20 | 1 of 6 | +0% |
| Persistent external memory stores | 14 of 31 | 8 of 20 | 0 of 5 | -3% |
| Multi-agent debate & consensus | 11 of 24 | 11 of 16 | 1 of 2 | -1% |
| Code-as-action / programmatic tool calling | 13 of 31 | 11 of 16 | 0 of 4 | -3% |
| Observation representation & perception tools | 18 of 46 | 11 of 21 | 0 of 7 | -2% |
| Fixed few-shot demonstrations | 8 of 17 | 5 of 9 | 3 of 5 | +0% |
| Adaptive re-planning | 4 of 7 | 1 of 2 | 1 of 1 | +10% |
| Single-shot retrieval-augmented generation | 13 of 35 | 13 of 23 | 1 of 2 | -8% |
| Agentic search / retrieval as a tool | 9 of 23 | 2 of 10 | 3 of 5 | -6% |
| Production / open coding-agent harness | 21 of 74 | 14 of 31 | 3 of 13 | -5% |
| Tool / skill retrieval & selection | 4 of 9 | 3 of 6 | 3 of 4 | +0% |
| Preference optimisation | 5 of 14 | 4 of 6 | 0 of 1 | -1% |
| Model-level efficiency & parameter editing | 3 of 7 | 2 of 3 | 1 of 1 | +0% |
| Direct / zero-shot prompting (no scaffold) | 9 of 62 | 7 of 25 | 1 of 2 | -17% |
Model product lines, within each maker
Each line against the same maker's other lines, only in experiments that ran both.
OpenAI
| Product line | Scored higher | Typical change |
|---|---|---|
| GPT-5 | 48 of 54 | +20% |
| o1 | 8 of 11 | +14% |
| GPT-4o | 24 of 48 | +1% |
| GPT-4 | 9 of 15 | +5% |
| GPT-5 Codex | 2 of 2 | +13% |
| GPT Sol | 4 of 6 | +5% |
| GPT Terra | 2 of 3 | +11% |
| GPT Astra | 1 of 1 | +8% |
| GPT-4.5 | 1 of 1 | +5% |
| o3 | 5 of 12 | +0% |
| GPT-4.1 | 6 of 16 | -10% |
| o4 mini | 3 of 9 | -19% |
| GPT-4.1 nano | 1 of 2 | -1% |
| GPT-4o mini | 5 of 25 | -11% |
| GPT-5 mini | 6 of 33 | -8% |
| o3 mini | 2 of 8 | -1% |
| GPT Luna | 1 of 4 | -4% |
| GPT-OSS | 1 of 5 | -25% |
| GPT-4.1 mini | 1 of 6 | -32% |
| GPT-3.5 | 2 of 19 | -16% |
| GPT-5 nano | 1 of 8 | -13% |
| o1 mini | 0 of 5 | -3% |
| GPT-5 Pro | 0 of 1 | -48% |
Anthropic
| Product line | Scored higher | Typical change |
|---|---|---|
| Claude Opus | 28 of 42 | +5% |
| Claude Sonnet | 23 of 47 | +0% |
| Claude Fable | 1 of 2 | +0% |
| Claude Haiku | 1 of 18 | -12% |
| Product line | Scored higher | Typical change |
|---|---|---|
| Gemini Pro | 24 of 33 | +21% |
| Gemini Flash-Lite | 4 of 8 | -9% |
| Gemini Flash | 10 of 30 | -11% |
| Gemma | 2 of 6 | -27% |
Alibaba
| Product line | Scored higher | Typical change |
|---|---|---|
| Qwen Max | 3 of 3 | +57% |
| Qwen VL | 3 of 3 | +32% |
| Qwen Plus | 2 of 2 | +13% |
| Qwen 3 | 8 of 15 | +4% |
| Qwen Coder | 2 of 6 | -18% |
| Qwen 2 | 1 of 9 | -24% |
| Qwen Flash | 0 of 2 | -8% |
DeepSeek
| Product line | Scored higher | Typical change |
|---|---|---|
| DeepSeek V4 Pro | 6 of 7 | +12% |
| DeepSeek V3 | 5 of 11 | +0% |
| DeepSeek R1 | 4 of 10 | -2% |
| DeepSeek V4 Flash | 1 of 5 | -11% |
| DeepSeek R1 Distill | 0 of 1 | -64% |
Z.ai
| Product line | Scored higher | Typical change |
|---|---|---|
| GLM 5 | 2 of 2 | +7% |
| GLM 4 | 1 of 2 | -1% |
| GLM Flash | 0 of 1 | -9% |
| GLM Air | 0 of 1 | -3% |
xAI
| Product line | Scored higher | Typical change |
|---|---|---|
| Grok 4 | 2 of 2 | +12% |
| Grok 3 | 1 of 1 | +1% |
| Grok Fast | 0 of 2 | -8% |
| Grok mini | 0 of 1 | -1% |
Meta
| Product line | Scored higher | Typical change |
|---|---|---|
| Llama 3 | 1 of 1 | +84% |
| Llama 4 | 0 of 1 | -46% |
Data
License and credit
Please cite the original paper when using a specific result or quote, and respect its license. Please credit HyperAgility with a link when reusing the rankings, categories, statistics or data: "Source: HyperAgility (hyperagility.org), based on results from the cited papers." The data is CC BY 4.0, except paper titles, abstracts and quotes, which belong to their authors. The code is MIT.
Built by Corbulo.