AI-written unit tests: 15.6% of failures came from wrong expected results

An arXiv preprint analyzed 14,320 Java unit tests written by four AI models under four prompting styles. Incorrect assertions caused 15.6% of test failures, and the tests rarely checked null, negative or empty inputs, showing a bias toward the happy path. The gaps appeared even in simple functions.

Photo: Balcer~commonswiki / Wikimedia Commons, CC BY 2.5 (cropped)

Sources

• Research arXiv（査読前） (2026-10-23)
  Beyond Coverage and Mutation: A Blind-Spot Analysis of LLM-Generated Unit Tests

Editorial policy ・ Terms of use