Your browser does not support JavaScript! Skip to main content
Free 30-day trial Customer portal Careers DO-178C Handbook
 
Rapita Systems
 

Industry leading verification tools & services

Rapita Verification Suite (RVS)

  RapiTest - Unit/system testing   RapiCover - Structural coverage analysis   RapiTime - Timing analysis (inc. WCET)   RapiTask - Scheduling visualization   RapiCoverZero - Zero footprint coverage analysis   RapiTimeZero - Zero footprint timing analysis   RapiTaskZero - Zero footprint scheduling analysis

Multicore verification

  MACH178   Multicore Timing Solution   RapiDaemons

Services

  V & V Services   Qualification   Training   Tool Integration  Support

Industries

  Aerospace (DO-178C)   Automotive (ISO 26262)   Space

Other

  RTBx   Mx-Suite   Software licensing   Product life cycle policy  RVS development roadmap

Latest from Rapita HQ

Latest news

Danlaw Acquires Maspatechnologies - Expanding Rapita Systems to Spain
Rapita co-authored paper wins ERTS22 Best paper award
A look back on Rapita's Multicore DO-178C training in Huntsville
RVS 3.17 Launched
View News

Latest from the Rapita blog

Why mitigating interference alone isn’t enough to verify timing performance for multicore DO-178C projects
There are how many sources of interference in a multicore system?
Supporting modern development methodologies for verification of safety-critical software
Flexible licensing software fit for modern working
View Blog

Latest discovery pages

do178c DO-178C Guidance: Introduction to RTCA DO-178 certification
matlab_simulink MATLAB® Simulink® MCDC coverage and WCET analysis
code_coverage_ada Code coverage for Ada, C and C++
amc-20-193 AMC 20-193
View Discovery pages

Upcoming events

Aerospace Tech Week Europe 2023
2023-03-29
Certification Together International Conference
2023-05-10
View Events

Technical resources for industry professionals

Latest White papers

DO178C Handbook
Efficient Verification Through the DO-178C Life Cycle
A Commercial Solution for Safety-Critical Multicore Timing Analysis
Compliance with the Future Airborne Capability Environment (FACE) standard
View White papers

Latest Videos

Efficient DO-178C verification - WCET analysis
Efficient DO-178C verification - Code coverage
Efficient DO-178C verification - Functional testing
SCADE Test video thumbnail
Complementary DO-178C verification with Ansys(R) SCADE Test(TM) and RVS
View Videos

Latest Case studies

Supporting ISO 26262 ASIL D software verification for EasyMile
RapiCover’s advanced features accelerate the certification of military UAV Engine Control
Front cover of whitepaper collins
Delivering world-class tool support to Collins Aerospace
View Case studies

Other Downloads

 Webinars

 Brochures

 Product briefs

 Technical notes

 Research projects

Discover Rapita

Who we are

The company menu

  • About us
  • Customers
  • Distributors
  • Locations
  • Partners
  • Research projects
  • Contact us

US office

+1 248-957-9801
info@rapitasystems.com
Rapita Systems, Inc.
41131 Vincenti Ct.
Novi
MI 48375
USA

UK office

+44 (0)1904 413945
info@rapitasystems.com
Rapita Systems Ltd.
Atlas House
Osbaldwick Link Road
York, YO10 3JB
UK

Spain office

+34 930 46 42 72
info@rapitasystems.com
Rapita Systems S.L.
Parc UPC, Edificio K2M
c/ Jordi Girona, 1-3, Office 306-307
Barcelona 08034
Spain

Working at Rapita

Careers

Careers menu

  • Current opportunities & application process
  • Working at Rapita
Back to Top

What are the overheads of code coverage? Using "Doom" as an example

Breadcrumb

  1. Home
  2. Blog
  3. What are the overheads of code coverage? Using "Doom" as an example
2015-05-01

One question that we are frequently asked is what instrumentation overhead RapiCover introduces. Achieving low overhead instrumentation is recognized by our customers as a key strength of RapiCover. To measure structural code coverage of embedded software, or for that matter any software, code coverage tools like RapiCover use instrumentation (additional code that records what code has been executed).

Raspberry PiRaspberry Pi

In practice, this is a question that we tend to answer as "it depends", which, although true, isn't a particularly helpful answer. Instrumentation overhead depends on three things:

  • The "shape" and size of the software to be measured. The complexity of the software will have an impact on how much instrumentation is required (for example, when measuring decision coverage, linear code requires less instrumentation than code featuring lots of if-statements).
  • The type of coverage required. In the world of high-integrity aerospace and automotive systems, this is driven by the software integrity level/development assurance level of the software. Closely related to this is the density of instrumentation needed to measure the required type of coverage. Some types of coverage (for example, function coverage) require very little instrumentation, whereas others (such as MC/DC) require more.
  • The overhead of each instrumentation point. This is affected by the approach that is taken for implementing the instrumentation code.

As you might expect, the first of these factors is entirely within the realm of the application developer. In addressing the second factor, we have implemented a number of optimizations within RapiCover which minimize the instrumentation density. How we address the third factor is a matter of some interest. RapiCover, like our timing analysis tool RapiTime, provides an "open interface" instrumentation library. We provide relatively lightweight, "out-of-the-box" generic instrumentation code. This gives a good starting position for our users – enough to prove the coverage process. Once the coverage process works, the instrumentation can be optimized.

To illustrate the benefits of a well-optimized instrumentation library, we are using the classic game "Doom", compiled for the Raspberry Pi – an ARM-based mini-computer typical of many embedded systems. We set the game up to be controlled by a demo file (a recording of a player's moves, acting as a test vector). We removed any parts of the code that rely on real-world timing, so that the game runs as quickly as possible. The result is a version of the game that requires no user input, instead playing through the demo recording as quickly as possible. The demo ("Doom Done Quick") is a play-through of the game, visiting every level. The following video shows the demo being played back at normal speed.

When played in real time, the play-through takes 1181 seconds. On the Raspberry Pi, running as quickly as possible, the play-through takes only 220 seconds. We added statement and decision coverage to the example code. With our generic instrumentation, the play-through takes 329 seconds – 49% slower than the uninstrumented version (220 seconds). Replacing the generic version with a highly-optimized, target-specific instrumentation library, we find that the play-through time is even less. With further optimization, we were able to reduce the overhead to 3.2% (227 seconds). If the overheads introduced by code coverage are a concern (and in embedded systems, this is often the case), the per-instrumentation point cost is likely to have the single largest impact. As the Doom example shows, the ability to optimize instrumentation points can be exploited to achieve impressively low overheads.

DO-178C webinars

DO178C webinars

White papers

DO178C Handbook Efficient Verification Through the DO-178C Life Cycle
A Commercial Solution for Safety-Critical Multicore Timing Analysis
Compliance with the Future Airborne Capability Environment (FACE) standard
5 key factors to consider when selecting an embedded testing tool

Related blog posts

False positive and false negative in software testing

.
2019-05-22

1000 conditions per decision ought to be enough for anybody

.
2019-01-28

Code coverage without instrumentation

.
2018-10-18

Merging coverage data from multiple test runs

.
2017-01-31

Pagination

  • Current page 1
  • Page 2
  • Page 3
  • Page 4
  • Page 5
  • Page 6
  • Next page Next ›
  • Last page Last »
  • Solutions
    • Rapita Verification Suite
    • RapiTest
    • RapiCover
    • RapiTime
    • RapiTask
    • MACH178

    • Verification and Validation Services
    • Qualification
    • Training
    • Integration
  • Latest
  • Latest menu

    • News
    • Blog
    • Events
    • Videos
  • Downloads
  • Downloads menu

    • Brochures
    • Webinars
    • White Papers
    • Case Studies
    • Product briefs
    • Technical notes
    • Software licensing
  • Company
  • Company menu

    • About Rapita
    • Careers
    • Customers
    • Distributors
    • Industries
    • Locations
    • Partners
    • Research projects
    • Contact
  • Discover
    • AMC 20-193
    • What is CAST-32A?
    • Multicore Timing Analysis
    • MC/DC Coverage
    • Code coverage for Ada, C & C++
    • Embedded Software Testing Tools
    • Aerospace Software Testing
    • Automotive Software Testing
    • Certifying eVTOL
    • DO-178C
    • WCET Tools
    • Worst Case Execution Time
    • Timing analysis (WCET) & Code coverage for MATLAB® Simulink®

All materials © Rapita Systems Ltd. 2023 - All rights reserved | Privacy information | Trademark notice Subscribe to our newsletter