Hardware acceleration features that make real-time hard - instruction caches

Continuing our series on how the presence of advanced hardware features in modern processors makes it more difficult to establish the worst-case execution time (WCET) of an application, this week we examine the issues surrounding the use of instruction caches in CPUs and the effect that this has on WCET in real-time systems.

What is cache memory and how does it work?

Caches are small amounts of high-speed memory located in close proximity to the CPU, in which the instructions or data immediately required by the CPU can be accessed quickly.

Cache memory is used in modern CPUs as a way to reduce the time taken to access information that is stored in memory, particularly in situations where the memory access time is significantly greater than the time the CPU takes to execute the instruction.

Although in the majority of Von-Neumann architectures there is a single unified cache, for the purposes of WCET analysis the caching of CPU instructions and data are often considered separately, as they give rise to different problems in the analysis process.

How does instruction cache use affect timing analysis?

The primary issue is that the cache causes the system to exhibit a wide range of execution times for a particular operation.

In the case of an instruction cache, the execution time of an instruction will depend on whether that instruction is present in the cache or whether it must first be fetched from the main memory.

The process of fetching an instruction from memory when it is not present in the cache (known as a "cache miss") causes a delay in the execution while the data is retrieved, which does not occur if the instruction is already present in the cache (a "cache hit").

In order to increase the performance in typical systems, cached data is divided into blocks consisting of a small sequence of instructions. When a cache miss occurs, an entire block will be transferred into the cache. With sequential instructions, this behaviour reduces the likelihood that the next instruction will result in a cache miss, increasing the overall system performance.

Instruction caches and WCET

When considering the worst-case execution time of the system, it is necessary to consider the longest execution time for each instruction.

Owing to the amount of potential variability in most systems (due to the effects of pre-emption, interrupts and external inputs) it is computationally infeasible to determine the state of the cache for each instruction, so it is generally assumed that each instruction executed may give rise to a cache miss.

For each instruction executed, the worst-case execution time is therefore significantly greater than the average execution time. This introduces a large degree of pessimism into the final WCET estimate for the whole system.

Handling instruction caches with RapiTime

We are often asked how RapiTime handles cache effects in real-time systems. One of RapiTime's advantages is that it uses observed data gathered during of the execution of the system software in its target environment, so it's possible to see exactly how caches affect the observed execution of the system, compared with a system with no cache (or where cache is deactivated)

RapiTime contains a number of analysis enhancements that can help to reduce the pessimism of the calculated WCET value. To ensure that the measured times are not optimistic, every section of code that can execute not-in-cache should be tested from not-in-cache at least once. The information provided in the RapiTime report can be used to increase the confidence that the testing strategy is sufficient to adequately exercise the code.