FAE Diary: A stormy AdaTest integration

In our FAE Diaries, you can follow the stories of Rapita Field Application Engineers on their adventures delivering our solutions to customers across the globe.

Steve Kerr visited a very wet part of the UK (courtesy of storm Dennis) to provide RapiTest training (testing and integration) and create an RVS/AdaTest integration to allow our customer to run their existing AdaTest tests with RVS.

To start the RVS training, I helped our customer to create RapiTest tests, and later on, helped them create their own host-based RVS project integrations. Once the team had gained confidence with these steps, we looked at creating an AdaTest RVS project.

To keep the scope of the AdaTest integration as simple as possible, we chose to start with a simple '2-package' example called Mandelbrot. This is an AdaTest test build that is structured similar to the customer’s 'real' AdaTest builds.

First, we ran the AdaTest example using an AdaTest installation on a Windows XP VM and it didn't render well with the 4k monitor we were using. We had to scale the RDP window to 175% just to be able to read the text.

AdaTest on XP

I examined the build system and realized that each UUT’s (unit under test) AdaTest test was compiled as a separate build with subprogram names that were the same in each build. This prevented us from creating an aggregate build system that could build multiple tests into a single RVS build unless we could make significant modifications to all test source files, which we didn't want to do if we could avoid it! Furthermore, the way that AdaTest manages stubs meant that we wouldn't be able to test a UUT that was also stubbed from another UUT in the same build. To overcome these limitations, we decided to start by working with an RVS/AdaTest integration on a single subprogram/test. We followed the AdaTest integration instructions provided with RVS 3.11 and set up an integration without needing to make any significant changes to the build system.

We ran the 'Prepare' stage and got a strange error from adains that seemed to be coming from the underlying ASIS analysis tool. The error was found while parsing a file that we didn't expect to be getting built – the combined 'UTH' sub-package spec and body from the UUT package that contained the test body and main entry point function. In the log, this error seemed to be happening before the gprbuild build system was being called. After much head scratching, I realized that this was caused by the call to gnatchop that was being used, prior to calling gprbuild, to break the test harness code up into separate files. gnatchop itself is not wrapped, but it calls gcc, which is wrapped. After consulting with the experts back at base, I set RVS_ENABLED to 0 in the batch build script prior to the call to gnatchop and set it back to 1 before the call to gprbuild.

We then built and ran the tests and cross-checked the results from those obtained when running the same test with AdaTest to make sure that they matched, which they did! The customer was very pleased with the solution, especially as they had also learned how to use rvsdriver to script the building and running of multiple tests. I informed them that I would see if we can improve the scripting of running these tests, although not to expect too much as the AdaTest stubs are a major limitation on compiling more than one test in a single build.

From the first day of the integration, the participants were impressed that RapiCover picks up on MC/DC coverage for Boolean expressions in assignment statements and not just those embedded in if/while etc., as AdaTest’s BOE coverage doesn’t. They were also impressed that RapiTest can run multiple tests in sequence in a single build, as this isn't possible with AdaTest.

Steve Kerr

Subscribe to our newsletter

Sign up to receive news updates and exclusive event invitations from Rapita Systems.

Subscribe »