Qualification - why is it?
Industry standards such as KT-178 or ISO 26262 describe processes for building robust software. If you follow these descriptions, the creation of such software will turn into a bureaucratic hell that will last forever. But there is software that can automate a significant part of these processes. Such software is called a tool. And if you use a tool, then it must be reliable (ISO 26262 even introduces the term “confidence in the tool”). To confirm the reliability of the instrument, its qualification is carried out.
Qualification - theory
Industry standards have a concept of security level. In different standards, they are called differently: Software Level in CT-178, Vehicle Safety Completeness Levels in ISO 26262. And for development tools, tool qualification levels (CT-178) or tool classification levels (ISO 26262) are used. These levels are assigned based on the criticality of the tools - the more influence a tool has on development, the higher skill level it will be assigned. In this case, one of the main criteria for determining the influence of a tool is the measure of its influence on the resulting software.
Consider a source code generator and a static code analyzer as examples. The generated code goes into the firmware of the device, which will be installed on board an aircraft or car. Thus, the code generator has a direct impact on the resulting software. Since a code generator is a complex thing, and can generate code with errors, strict requirements are imposed on the quality of this code generator and the level of its qualifications will be maximum. Another thing is a static analyzer, the result of which does not get into the onboard software and the degree of its influence is minimal. Therefore, the skill level for a static analyzer will be lower than for a code generator.
And the level of qualification directly affects labor costs: for example, for aviation, to qualify a tool for the highest level of KT-178S, 76 control measures are required, and for the lowest level - only 14.
Another important point is that the qualification of tools is not carried out by the developer of the tool, but directly by the software developer, and the qualification must be carried out for each project!
Qualification - Practice Notes
As mentioned in the theoretical part, tool qualification is a costly process, but it is simplified in several ways:
- Support of the qualification process by tool manufacturers (vendors)
- Instrument qualification guidelines from standards
Let's consider them in order. First of all, the vendors themselves supply the tool qualification support packages. Second, industry standards provide guidance on qualifying tools that can reduce workload.
I would like to dwell on the second point. Let's look at the code generator again. The code generated with its help must still be tested, the code coverage will be collected, its analysis is carried out, that is, verified.
Industry standards tell us that if we qualify the tools that are responsible for code verification, then the qualification of the code generator itself will not be needed. Thus, the standards themselves provide guidance that you can follow to significantly reduce the cost of qualifications.
For the qualification of verification instruments, the standards prescribe demonstrating their behavior under normal conditions. In practice, it looks like this:
- The vendor supplies a set of document templates, regulations and benchmarks for the instrument and their reference results.
- You fill out document templates and run the provided tests in your environment.
- The results of the tests you run are compared with the benchmarks, and if there is a discrepancy in the results, you eliminate the discrepancy.
It is worth noting that the described process also takes a lot of time and, like all development, is carried out in cooperation with the certifying authorities.
MathWorks tools and their qualifications
Tools such as Simulink, DSP Toolbox, Control System Toolbox are the industry standard for developing control systems, digital signal processing. It is not surprising that they are used in aviation, automotive and other industries. The developed models are used to generate C / C ++ code that travels and flies. Naturally, developers are faced with the question of tool qualifications. And the qualification of MathWorks tools for KT-178S is carried out for model and code verification tools:
- Simulink Check
- Simulink Coverage
- Simulink Requirements
- Simulink Design Verifier
- Simulink Test
- Simulink Report Generator
- Polyspace Bug Finder
- Polyspace Code Prover
- Simulink Code Inspector
And for ISO 26262, certificates are supplied for:
- Simulink Check
- Simulink Coverage
- Simulink Requirements
- Simulink Design Verifier
- Simulink Test
- Simulink Report Generator
- Polyspace Bug Finder
- Polyspace Code Prover
- Embedded Coder
- HDL Coder
- PLC Coder
Depending on the industry, DO Qualification Kit for Aviation or IEC Certification Kit for the automotive, railroad and other industries are provided .
Instead of conclusions
This article has provided a high-level overview of the industry-standard tool qualification process. In the course of writing this article, I was guided by the following sources:
- R-330, "Software Tools Qualification Guide", in particular:
- p. 2.0 Assignment of tool qualifications
- p. 3.1. Qualification levels
- Reference D question D7
- GOST R ISO 26262-8, Chapter 11, "Confidence in using tooling software"
If you are interested in how systems of increased reliability are created, then write in the comments what you need to consider in the following articles.