Does DO-178C require object code structural coverage?

If you are developing software to Level A for DO-178B/C, your code has to undergo extremely rigorous structural coverage analysis for the purposes of certification. This includes examining both source and object code.

Code coverage testing aims to ensure that all of your source code can be traced back to requirements. You can achieve this by running your tests (which should be derived from requirements), and showing that you have achieved complete code coverage. In DO-178B/C this is slightly complicated by an additional requirement: the need to identify and test any object code which may have been generated by the compiler and isn’t directly traceable to the source code.

The wording of DO-178B for Level A structural coverage testing can lead to the impression that the best way to both comply with the standard, and get adequate coverage results, is to perform your structural coverage analysis directly on the object code. However, DO-178C and the FAA CAST 12 paper [1] on the subject makes it clear that this is a misconception: object code analysis is not a simple replacement for source code coverage analysis.

In fact, neither structural analysis at the source or object code level will, on their own, provide you with a full solution for Level A. You will always need some analysis at both source and object code levels. However, there is a choice as to which you perform the structural coverage testing on. It is important to be aware that the choice you make leads to different additional material being needed to make a certification case. I’ve summarized these below:

1. When performing coverage testing on source code, up to and including MC/DC, you will need to analyze any object code that cannot be traced to the source (e.g. processor specific instructions inserted by the compiler). Analysis of the source code has the advantage of clear visibility of the source code coverage with faster results, that can be run and rerun earlier in the design process, but a safety case for the use of source code instrumentation is needed. If data to make this case can be supplied by the coverage analysis tool provider, this can reduce the cost of making such a case significantly.
2. When performing coverage testing on the compiled object code, you will need to analyze the mappings from the object code to source code decisions, and show that the test suite exercises all the original source code. This is necessary, as although full object code coverage may have been achieved, if the compiler has optimized the original source, we may not be certain that all the source code has a related test case (see the simplified diagram below!). If a mapping from the object code to source code can’t be provided, you will need to perform additional MC/DC testing.

In summary, the certification criteria for Level A code coverage testing can be met by both source code and object code analysis, with source code coverage analysis offering advantages in faster analysis and easier traceability between source code and results. Using either type of analysis, a certification case must be made by providing evidence to support the analysis. For source code analysis, you may be able to obtain some of this evidence from the tool provider, potentially reducing the cost of certification.

[1] Position Paper CAST-12: Guidelines for Approving Source Code to Object Code Traceability 2002.