Application Coverage Is the Most Powerful Metrics Activity You’re Not Doing

Code coverage is a first step in an overall software quality process that enables you to begin understanding where risk is lurking in your application. Furthermore, the methods in which coverage is measured matters—different activities measure different dimensions of the application.

Enter application coverage, which expresses the degree to which the application’s source code is exercised by all testing practices. Application coverage provides extremely powerful insight into risk because it exposes untested or under-tested parts of the application. Before getting into the details of application coverage, let’s discuss how code coverage is usually measured.

What’s Wrong With How I Normally Measure Coverage?

Dev/test teams typically lean on unit testing as the primary vehicle for driving coverage. Unit testing also provides many other valuable benefits, such as:

• Instant and automatic proof that the code works anytime, anywhere
• Confidence to refactor code, resulting in improved design and better maintainability
• Almost-instant feedback if something breaks
• The ability to validate the code prior to checking it in (because tests can run locally and independently of the continuous integration infrastructure)
• Enables development teams to fix bugs while the code is top-of-mind, instead of debugging for hours/days as the code base ages

To be clear, we are huge fans of unit testing and if time weren’t an issue, we would run the full regression suite after every change to monitor coverage and ensure the safety, security, and reliability of the application. Obviously, no one has the time or resources to implement this level of testing. Besides, measuring coverage during functional and manual test execution is also important for getting a holistic view of how well the application is covered.

The bottom line is that measuring coverage during unit test execution is great, but also measuring coverage during functional and manual testing is even better.

Why You Should Add Application Coverage to Dev/Test Mix

A critical metric for assessing ‘release readiness’ is application coverage. Application coverage is the aggregation and correlation of code coverage from various testing techniques. By aggregating code coverage from all testing practices, organizations can have better visibility into risky areas of the application.

No Time for Testing? No Problem

In addition to enabling more accurate quality and coverage metrics, application coverage can also help organizations focus testing efforts when time constraints limit their ability to run the full suite of manual regression tests. In exploratory testing, for example, testers know that the code around certain functionality has changed and quickly attempt to identify unexpected behavior by performing manual testing around the changed functionality. Capturing coverage data on the running application during exploratory testing potentially increases the value of this approach because it enables a better understanding of what code the tester actually touched.

A Development Testing platform such as Parasoft DTP can reduce the dependency on arbitrary exploratory testing by providing teams with a better understanding of what changed, what areas are potentially impacted, and what needs to be retested. Exploratory testing then becomes an additional layer of testing used to fill in any gaps.

No Single Metric or Activity is a Defect Prevention Panacea

It’s important to note, however, that application coverage alone is not a credible measure of application quality. Coverage data should also be expressed within the context of other software quality metrics, such as static analysis violations, unit test results, etc., in order to provide a roadmap for preventing software defects. The screenshot below shows a view in Parasoft Development Testing Platform (DTP) with coverage data, unit test results, functional testing results, static analysis violations, and other metrics. This level of granular detail helps managers understand the current state of risk in the application, as well as the impact of recent changes on quality metrics.

But Wait, There’s More

We could go on and on about the benefits of application coverage and how Parasoft DTP merges and correlates the data with coverage from other testing activities to provide a bevy of software quality benefits. Instead, we’ll save ourselves some time and direct you to our on-demand webinar, Release = Merged, Correlated Coverage (MC2). We discuss the challenges that development teams face and how application coverage is the driving force for helping teams meet their release objectives.

If you liked what you’ve read and are interested in seeing our application coverage capabilities first-hand, contact us and we’ll be happy to schedule a demo.