Coding agents fully delivered at most 14% of enterprise product specs in a new test

A benchmark of 43 tasks in mature enterprise codebases, checked against 750 expert-audited points, measures whether coding agents fulfill full product requirement documents. Across 11 model and tool setups, the best complete-delivery rate was 14.0%, by OpenAI's GPT-5.6 Sol; missing integration was a main cause.

Photo: 江上清风1961 / Wikimedia Commons, CC BY 3.0 (cropped)

Sources

• Research arXiv（査読前） (2026-10-19)
  When Coding Agents Outpace Review: Evaluating PRD-Driven Development in Mature Enterprise Repositories

Editorial policy ・ Terms of use