Multicore Timing Solution

When developing safety-critical applications to DO-178C (CAST-32A) guidelines or ISO 26262 standards, there are special requirements for using multicore processors. Evidence must be produced to demonstrate that software operates within timing deadlines.

The goal of multicore timing analysis is to produce execution time evidence for these complex systems. In multicore processors, multiple cores compete for the same shared resources, resulting in potential interference channels that can affect execution time. Accounting for this interference and producing robust execution time evidence is a challenge addressed by Rapita’s Multicore Timing Solution.

Our CAST-32A Compliance packages comprise three components: a process, tool automation, and services.

Our multicore timing analysis process is a V-model process that we developed in line with DO-178 and CAST-32A. It follows a requirements-based testing approach that focuses on identifying and quantifying interference channels on multicore platforms. We help address CAST-32A objectives not related to multicore timing through engineering services.

The tools we have developed let us apply tests to multicore hardware (RapiTest) and collect timing data (RapiTime) and other metrics such as scheduling metrics (RapiTask) from them. We use RapiDaemons to create a configurable degree of traffic on shared hardware resources during tests, so we can analyze the impact of this on the application’s timing behavior.

Our CAST-32A Compliance services include tool integration, porting RapiDaemons, performing timing analysis, identifying interference channels, and others depending on customer needs.

No, but we are currently working on projects that will go through certification with the FAA and EASA.

Rapita is a recognized leader in multicore timing analysis, with multiple professional publications on the subject:

• Steven H. VanderLeest and Samuel R. Thompson, “Measuring the Impact of Interference Channels on Multicore Avionics”  to appear in the Proceedings of the 39th Digital Avionics Systems Conference (DASC), San Antonio, TX, Oct 2020.
• VanderLeest, S.H. and Evripidou, C., “An Approach to Verification of Interference Concerns for Multicore Systems (CAST-32A),” best papers of 2020 SAE Aerotech, to appear in SAE International Journal of Advances and Current Practices in Mobility.
• VanderLeest, S.H. and Evripidou, C., “An Approach to Verification of Interference Concerns for Multicore Systems (CAST-32A),” SAE Technical Paper 2020-01-0016, 2020, doi:10.4271/2020-01-0016.
• Steven H. VanderLeest, Jesse Millwood, Christopher Guikema, “A Framework for Analyzing Shared Resource Interference in a Multicore System,” Proceedings of the 37th Digital Avionics Systems Conference (DASC), London, Sep 2018.

Our multicore team technical leads have also served in the following professional organizations:

• Vice-chair for the Enterprise Architecture (EA-25) subcommittee on airworthiness for the Future Airborne Capability Environment (FACE) consortium, 2020.
• Chair for the “Multicore Assurance” session of the 39th Digital Avionics Systems Conference, 2020.
• Chair for the “Integrated Modular Avionics” track of the 37th Digital Avionics Systems Conference, 2018.

In the CAST-32A position paper published by the FAA, an interference channel is defined as "a platform property that may cause interference between independent applications". This definition can be applied to a range of ‘platform properties’, including thermal factors etc.

Of these interference channels, interference caused by the sharing of certain resources in multicore systems is one of the most significant in terms of execution times. Interference based on shared resources may occur in multicore systems when multiple cores simultaneously compete for use of shared resources such as buses, caches and main memory.

Rapita’s Multicore Timing Solution analyzes the effects of this type of interference channel.

A very simple example of a shared resource interference channel is as follows:

In this simplified example, tasks running independently on the two cores may need to access main memory simultaneously via the memory controller. These accesses can interfere with each other, potentially degrading system performance.

This depends on the platform, project needs, and whether we have already performed analysis on a similar hardware platform previously. Our solution includes an initial pilot phase in which we study the system and estimate the amount of time needed for subsequent phases. Typical projects run for between 2 and 12 months, depending on the scope of the analysis and complexity of the system.

Some of the multicore systems that we’ve worked with are listed in our FAQ “Which hardware architectures can you analyze?”.

If we have already worked on a similar multicore platform to yours, it may take less time to perform hardware analysis for your platform.

As per CAST-32A, each interference channel on the system must be analyzed to ensure that its effects on timing behavior have been mitigated. Some channels may be eliminated by architectural analysis or because they are deactivated for your system. For all channels that have not been eliminated, CAST-32A requires that multicore timing analysis be done to ensure that software operates within its timing deadlines. For each channel, one must provide analysis test reports and a summary to show that the set of tests is complete – Rapita provides this service to show completeness. Note that CAST-32A includes objectives other than testing, such as planning objectives. We provide templates to help you complete these.

RTOS vendors may provide partitioning mechanisms for their multicore processors, but these do not guarantee the complete elimination of interference. Instead, they are designed to provide an upper limit on the interference, sometimes at the expense of average-case performance.

In aerospace, these partitioning mechanisms may be referred to as ‘robust partitioning’. CAST-32A (the FAA’s position paper on multicore processors in avionics) identifies allowances for some of the objectives if you have robust partitioning in place – but it is still necessary to verify that the partitioning is indeed as robust as claimed.

From a certification standpoint, regardless of the methodology behind the RTOS vendor’s approach to eliminating interference, the effectiveness of the technology needs to be verified.

We’re completely flexible, and we understand that different customers have different needs. As such, you can purchase any component of our multicore timing analysis solution separately if that’s what you need. This includes, but is not limited to:

• Tool licenses (RapiTest, RapiTime, RapiTask)
• Services to integrate automation tools to work with your multicore system
• RTBx hardware to collect trace data from your multicore system
• Generic libraries of RapiDaemons
• Services to port RapiDaemons to your multicore system
• Services to perform multicore timing analysis

For safety reasons, WCET will always be somewhat pessimistic. However, techniques that work well for single-core systems risk generating a WCET that is unreasonably large when applied to multicore systems, because the effects of contention can become disproportionate. The objective, therefore, is to calculate a value that is plausible and useful, without being optimistic. Optimism in relation to WCET is inherently unsafe.

It is not enough to identify how sensitive an application’s tasks are to different types and levels of interference; it is also necessary to understand what degree of interference a task may suffer in reality. It is possible to lessen the pessimism in WCET analysis by viewing the processor under observation through this paradigm.

The degree to which we can reduce pessimism is dependent on how effectively we can analyze the system. Factors influencing this include:

• The overhead of the tracing mechanism (which affects depth of instrumentation)
• The availability and reliability of performance counters
• The availability of information regarding other tasks executing on the system
• The quality of tests that exercise the code

To analyze how a specific task is affected by contention on a specific resource, we need to be able to synchronize the execution of the task with the execution of RapiDaemons (the applications that generate contention on the resource).

Usually it is highly desirable to have RTOS/HV support for enabling user-level access to performance counters. Additionally, context switch information is very valuable when performing timing analysis.

The maximum number of metrics we can collect depends on the performance monitoring unit(s) (or equivalent) on the hardware. An ARM A53, for example, lets us collect at least 30 metrics, but only access 6 in a single test. By running tests multiple times, however, we could collect all 30 metrics.

It is possible to collect a range of metrics by instrumenting your source code with the Rapita Verification Suite (RVS), including a range of execution time metrics:

• RapiTime: high-water mark and maximum execution times
• RapiTask: scheduling metrics such as periodicity, separation, fragmentation and core migration

It is also possible to collect information on events in your hardware using performance counters. The information we can collect depends on the performance monitoring unit(s) (or equivalent) of your system, but typically includes events such as L2 cache accesses, bus accesses, memory accesses and instructions executed. We can also collect information about operating system activity such as task switching and interrupt handling via event tracing or hooks.