AI-supported functions have long been a reality in regulated industries: from CT segmentation in medical technology to ADAS object recognition in vehicles to automated credit assessment. At the same time, the regulatory framework is becoming more stringent.
The EU AI Act will impose binding requirements on high-risk AI systems from August 2026. New standards such as ISO/PAS 8800 for automotive and updated FDA guidance for AI medical devices specify what manufacturers must demonstrate. For companies that develop or use AI components, one question is becoming urgent: How do you ensure that your AI not only works, but is also auditable?
Traditional software testing is based on a simple assumption: same input, same output. AI systems do not follow this principle. Their results are probabilistic and context-dependent. Two identical queries to the same model can return different but equally plausible answers. This poses concrete problems for testing teams:
These gaps are particularly significant in regulated industries. Anyone who cannot demonstrate how AI arrives at its results in a conformity assessment under the EU AI Act risks delays or rejections.
Teams that only address AI quality assurance shortly before the audit face a considerable backlog. This is because the required evidence relates not only to the finished model, but also to training data, explainability, and continuous monitoring.
The good news is that there are now proven approaches that are specifically tailored to the peculiarities of AI.
Metamorphic testing, for example, circumvents the problem of the missing test oracle by checking logical relationships between inputs and outputs rather than individual results. A rotated X-ray image should still be classified correctly. If not, this indicates a lack of robustness.
Explainability methods such as SHAP or LIME make AI responses comprehensible and provide the transparency evidence required by regulators.
Continuous model monitoring detects drift phenomena in productive operation before they impair model quality.
Bias testing with various test data sets ensures that AI systems do not systematically disadvantage any groups of people.
No single tool can solve all the challenges. Effectiveness comes from the interplay between a risk-based testing strategy, appropriate methods, end-to-end traceability, and a quality management system that meets regulatory requirements. It is precisely at this interface between technical testing know-how and regulatory expertise that it is decided whether AI quality assurance is viable or only works on paper.
Would you like to exchange ideas with experts about quality assurance for AI systems? At our Afterwork Exchange – Business Impact: AI, practitioners from various industries discuss current challenges and possible solutions.
No. AI models are subject to model drift and must be continuously monitored. The EU AI Act and industry-specific standards require post-market surveillance throughout the entire product life cycle.
Yes. In the future, AI medical devices must comply with both the MDR/IVDR and the EU AI Act. For high-risk systems, the full requirements will apply from August 2026.
Instead of checking exact results, logical relationships are tested: Does a classification remain consistent when inputs are slightly changed? This approach is particularly suitable for systems without a clear test oracle.
Not necessarily different ones, but additional ones. Existing ALM and CI/CD systems remain the basis. They are supplemented by AI-specific tools for bias testing, explainability, and drift monitoring.
All industries with high-risk AI systems: medical technology, automotive, financial services, and public administration. Regulatory requirements are converging across industries.
Firstname:
Lastname:
E-Mail Address:
Phone:
Subject:
Your message:
Yes, I consent to my personal data being collected and stored electronically. My data will only be used for the purpose of responding to my inquiry. I have taken note of the privacy policy.
You are currently viewing a placeholder content from OpenStreetMap. To access the actual content, click the button below. Please note that doing so will share data with third-party providers.
You need to load content from hCaptcha to submit the form. Please note that doing so will share data with third-party providers.
You are currently viewing a placeholder content from Hubspot Meetings. To access the actual content, click the button below. Please note that doing so will share data with third-party providers.
You are currently viewing a placeholder content from Google Maps. To access the actual content, click the button below. Please note that doing so will share data with third-party providers.