Nvidia's custom harness lifted Claude Opus 5 from a 30 percent to a perfect 100 percent score on the ARC-AGI-3 reasoning benchmark
The harness adds a supervising agent that redirects the main agent when it stalls or wanders off track
Databricks and OpenAI research point to the same trend: harness design, not model choice, increasingly drives agent performance and cost
Nvidia's new research suggests the software wrapper around an AI model, known as the harness, matters more than the model itself for tasks that require stringing together many decisions over time.
Using a custom harness built for memory handling and a supervising component. Researchers pushed Claude Opus 5 to a perfect score on ARC-AGI-3, an interactive benchmark of unlabeled 2D games. Without that harness, the same model scored just 30 percent, still the best raw score among tested models.
Long-horizon tasks, ones spanning many linked decisions rather than a single prompt response, remain one of the hardest problems in agentic AI. Microsoft research from April found that all 19 tested language models filled long documents with errors during extended editing tasks. Other models have deleted user files or turned to deceptive behavior while pursuing goals autonomously.
OpenAI ran its own tests after scoring under 10 percent on the same benchmark. Tweaking two harness settings tripled its scores, but no OpenAI model reached anywhere near Nvidia's 100 percent result. Nvidia's edge came from adding a supervising agent that acts like a manager, nudging the main agent away from dead ends or repeated paths.
Databricks reported similar findings in July, showing that harness choice, not model choice, can double AI costs even when the underlying model stays the same. Nvidia frames its results as evidence for open, modular harness design over relying on a single closed layer.