Three AI Tasks in Software Testing That Are Often Mixed Up

The next release is scheduled, and testing capacity is tight. Since AI started writing code alongside developers, software gets built faster, and one question is getting louder: How do quality assurance (QA) and evidence keep up? In regulated environments, every release must also be documented and justified. That turns software testing into a sourcing question: build testing expertise in-house, bring in external specialists, or combine both deliberately?

Before that question can be answered, it is worth taking a closer look at the phrase “AI in software testing”. Behind it usually sit three different tasks.

Why the Make-or-Buy Question Goes Nowhere Without This Separation

Anyone who does not separate the three tasks compares apples with oranges when deciding between building and buying. The request “We need more testing” can mean additional verification capacity, building up tooling and automation know-how, or a specialist topic for which nobody in the company has experience. Each of these tasks has different requirements for expertise, utilization and evidence.

A simple cost comparison, such as the salary of one employee against the day rate of an external specialist, does not provide a consistent basis for the calculation. The result is a decision on unclear ground: staff are tied up long-term for the wrong task, and releases become harder to justify convincingly. The article “Building Software Test In-House: The Hidden Bill” describes the costs of an in-house build in general.

Sources

• Becker, J., Rush, N., Barnes, E., & Rein, D. (2025). Measuring the impact of early-2025 AI on experienced open-source developer productivity (arXiv:2507.09089). METR.
• European Parliament and Council. (2024). Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act). Official Journal of the European Union.
• Sonar. (2025). Sonar data reveals critical “verification gap” in AI coding.