An early look at RapiTestFramework

May 9th, 2017

We have been hard at work recently developing RapiTestFramework, a new RVS tool that generates and runs unit, integration and system tests on embedded targets or host systems. RapiTestFramework injects test framework code, builds a test harness, runs this on the target and reports results.

RapiTestFramework has come along very well during its early development, and has been used by both the embedded software industry and ourselves (as part of our new software verification services). What's more, we have been working on kits to help you qualify the tool for use in your DO-178B/C or ISO 26262 projects. 

In this post, we thought we'd share with you some of the design decisions we made during the development of RapiTestFramework, which led to the strong position the tool is in today. 

Thinking of the future

All of our tools are built to meet the evolving needs of embedded software verification, and RapiTestFramework is no exception. Because these needs change over time, we built RapiTestFramework to be flexible from the outset.

We did this by spending the early design period concentrating on the features we need from the tools that parse source code and inject tests, based on real needs identified from our customers.  

By using a bottom-up approach and leaving questions about higher-level features such as the user-interface for later, we could spend the time needed to engineer a flexible tool that will be relevant for years to come. 

There are some really exciting features we can implement using RapiTestFramework, such as data and control coupling and identifying infinite loops in your source code before running it, but you'll have to wait until a future blog post for more on that.

Flexible test formats

The first step in running software tests is to write them, and the availability of powerful test authoring formats is a huge factor in overall testing efficiency. A key part of our initial design specification was to offer flexible options for you to write tests.  

We have developed two test formats specifically for RapiTestFramework, both a user-friendly spreadsheet format and a custom scripting language (Figure 1.). By looking at how our existing customers write their tests, we identified some of the main drivers of inefficiency in the process to guide the design of these formats.

  • Figure 1: RapiTestFramework test formats: spreadsheet
  • Figure 1: RapiTestFramework test formats: script

The spreadsheet format is compact, easily readable, and lets you write tests quickly, while the script format makes it easy to write very complex tests, including those with conditional logic.

An important part of overall development has been documenting both languages thoroughly so you can get the best from RapiTestFramework with minimal training.

In keeping with our desire to offer you flexible options, if you have existing tests, we can convert these into a format you can use with RapiTestFramework.  

Managing your projects

As part of the development of RapiTestFramework, we had the opportunity to rethink how we can help manage your test projects. This led us to make a major overhaul in the RVS workflow, which has taken a lot of complexity away from you, the user. 

With RapiTestFramework, you can configure and run a test project through a single user-interface, the new RVS Project Manager (Figure 2).

  • Figure 2. The RVS Project Manager: project creation
  • Figure 2. The RVS Project Manager: project configuration
  • Figure 2. The RVS Project Manager: viewing results

In many development environments, the RVS Project Manager can invoke your build system directly, so you can run your test project, collect and analyze results with only a few clicks.

You can also run RapiTestFramework from a command-line interface. Our new command-line tool rvsdriver makes it simple to run RapiTestFramework. This tool manages and drives all of our low-end tools, so you only need to provide a few command-line options to run your project.

We're really excited about the new RVS workflow. The RVS Project Manager and rvsdriver, as their names suggest, are designed to work with all RVS tools, and will roll out to RapiCover, RapiTime and RapiTask in the upcoming RVS 3.7 release. 

If you would like to receive more updates on RapiTestFramework, make sure you are signed up to our mailing list.  

Daniel Wright