Fresenius Medical Care

Automated validation of a continuous integration tool chain

The task

Since 1979, Fresenius Medical Care has played a key role in the medical success story of treating chronic kidney disease. The continuous development of products poses a particular challenge for software development, which must produce high-quality code in ever shorter release cycles. This would be virtually impossible without the concept of continuous integration. Continuous integration (CI) means that the software being developed is compiled, built, and then tested as often as possible. This provides developers with rapid feedback on changes that have been made.

In addition to the actual build environment, the required tool chain includes a range of software tools that perform various checks. These include tools for static and dynamic code analysis and the framework for automated unit tests. This tool chain must be validated for use in a regulated environment.

Fresenius Medical Care commissioned sepp.med gmbh to carry out the validation. sepp.med first conducted an inventory of the existing tools. This “inventory” resulted in a list of twenty tools that can be used depending on the project. Furthermore, sepp.med clarified the planned use of the CI tool chain in a series of stakeholder interviews. Among other things, it determined which programming languages are relevant, which tools are used, and what the use cases look like. These were documented graphically as UML diagrams and further detailed in activity diagrams.

At the same time, Fresenius Medical Care conducted a risk analysis (FMEA) and identified a series of measures. Together with the workflows determined in the interviews, the risk analysis formed the basis for the requirements specification for the tool chain.

Particular focus was placed on validating the MISRA C++ rules and the exceptions specified by Fresenius that must not lead to error messages.
Various scenarios were tested for the compilers to be validated, including configuration, termination in the event of an error, possible warning levels, and, of course, the generation of binaries. In addition, sepp.med specified workflow tests in the validation plan based on the identified processes.

Right from the start of the project, it was clear that some of the tests would need to be automated. Firstly, it would have been time-consuming and error-prone to assign the various test data “by hand” and evaluate the log files by literally “reading” them. Secondly, the amount of test data was unmanageable manually, as several hundred MISRA rules had to be verified. Thirdly, it was foreseeable that a large number of tests would have to be repeated. sepp.med evaluated various alternatives. The choice finally fell on the test automation framework Spock, the test execution tool gradle, and the programming language Groovy. The central control unit is the Jenkins server, which is used to check out and configure the test project.

The requirements, test cases, and test execution of the manual and automated tests by sepp.med were documented in the ALM tool Polarion. Finally, sepp.med created a validation report.

Automation

The test automation concept was successfully adopted in a follow-up project. This was based on the textual specification of the expected system behavior, as is common in behavior-driven development (BDD) and supported by Spock.

This reduced the checking of numerous MISRA rules to a simple, easy-to-understand scheme:

Given [precondition]
Expect [MISRA rule violation is detected]
Where [MISRA rule]

Using this BDD scheme, the basic scripts created can be recombined at any time, allowing new test cases to be specified and implemented with minimal effort.

Conclusion

Although – or rather because – the CI tool chain was not initially approved, Fresenius Medical Care and sepp.med consider the validation a success. On the one hand, they succeeded in avoiding errors caused by blind trust in tools. On the other hand, they laid the foundation for further automated validations. The automated tests have already been repeated several times since the completion of the first run.

There are now plans to automate the workflow tests as well. This will make it possible to fully test every new CI server within half a day before it is put into operation.

Fresenius Medical Care is the world's leading provider of products and services for people with chronic kidney failure, of whom around 3.2 million patients worldwide undergo regular dialysis treatment. The company cares for more than 320,000 patients in a global network of more than 3,700 dialysis clinics.

Fresenius Medical Care is the world's leading provider of dialysis products such as dialysis machines, dialyzers, and related disposable accessories, offering products and services along the entire dialysis value chain from a single source. With innovative products and treatment concepts of the highest quality, Fresenius Medical Care is continuously working to improve the quality of life of patients with kidney disease.

Fresenius Medical Care

Automated validation of a continuous integration tool chain

in collaboration with Fresenius Medical Care

The task

The task

The procedure

The factors for success

Requirements analysis

Modeling

Automation

The result

Conclusion

Get in touch with us!