Featured Webinar: MISRA C++ 2023: Everything You Need to Know | Watch Now
Code Coverage: Assess and Ensure Confidence in Your Testing
Parasoft’s solution for code coverage exposes the code in your application that has and has not been tested. Users can perform code coverage at various levels, function or unit, the entire file, or the entire application on target hardware or host environments. And you can aggregate the coverage from the various testing methods like unit testing and system testing.
What Is Code Coverage?
Structural code coverage is the identification of code that has been executed and logged for the purpose of determining if the system has been adequately tested.
Code coverage expresses the degree to which the application’s source code is exercised by all testing practices, including unit testing, manual testing, automated functional testing, and the like. It also:
- Enables more accurate quality and coverage metrics.
- Exposes dead code and untested or under tested parts of the application.
As a result, application coverage provides extremely powerful insight into risk. In most organizations, unit testing is the primary vehicle for driving coverage. And while unit testing is a valuable testing practice that also enables several process-oriented benefits, it is also expensive in terms of the expertise and time to create, manage, and maintain the tests.
Code coverage can be leveraged as part of the continuous integration (CI) process, as well as part of the developer desktop workflow. You can also perform advanced analytics on the source code and identify which unit tests need to be rerun based on code changes the developer has performed.
These code coverage tools are especially useful in industries for embedded development of safety- and security-critical applications where software systems cannot fail, or lives will be lost.
The thoroughness of the coverage in safety-critical systems depends on the application safety integrity level (SIL or ASIL) metric used in various industries and the development assurance level (DAL) commonly used in avionics. Thoroughness refers to the structural elements in code. These are typically broken down to the code statement, branch, modified condition decisions and with Parasoft, you can also drill down to a much finer level of coverage granularity, such as the object code or assembly language.
What Code Coverage Percent Is Ideal?
Code coverage percentage goals can be subjective in some cases. In other cases, they’re mandatory. When building applications that are safety-critical where failure may cause death, regulatory and industry standards require 100% structural code coverage.
For applications that aren’t safety-critical, code coverage tends to be left to the development organization to decide. The common comfort level for these types of circumstances is achieving a goal of 80% code coverage.
Top 6 Benefits of Code Coverage
Structural Code Coverage Tools
Deploy Parasoft’s suite of products to conduct structural code coverage of applications written in C, C++, Java, C#, and VB.NET.
8 Best Practices for Code Coverage
Obtaining the amount of code exercised during testing is a powerful metric in order to understand the level of risk remaining in the application. Here are some best practices to follow.
How to Get Started With Code Coverage
Parasoft code coverage within the IDE and DTP dashboard reporting and analytics solutions.
1. Become Familiar With Your Coverage Requirement Needs
To get started in collecting code coverage, become familiar with your coverage requirement needs. Maybe your industry and type of application don’t require any coverage metrics to be obtained, but you want to ensure or improve code quality, so 75% might be your initial goal.
If your industry requires compliance to a functional standard like DO-178C for avionics or ISO 26262 and ISO 21434 for automotive or IEC 62304 for medical, then determine if you need to obtain 100% code coverage or another percentage of code coverage is recommended. Also, know what structural code coverage type you need to satisfy. It could be the function, line, statement, block, call, path, decision, simple condition, MC/DC, object/assembly, or a combination of these.
2. Obtaining Code Coverage
One of the easiest ways to obtain code coverage is during implementation when engineers are creating unit test cases to test their code. With solutions like Parasoft C/C++test and Jtest,
- Users get a very short learning curve since it integrates right into your IDE and provides this functionality.
- QA teams can reuse these unit test cases for regression testing and/or as part of the build process in an automated CI pipeline workflow.
Be aware that instrumenting the code can cause code bloat and if you are testing on target hardware you may need to do partial instrumentation. Instrumented code may also alter the performance of the execution, so you may want to become acquainted with various optimization features available to you.
3. Collecting and Reporting Your Code Coverage Metrics
Collecting and reporting your code coverage metrics is crucial. There are several methods based on if your application is running on an embedded target, or a fully resourced system or server.
Ultimately, you’ll obtain or merge coverage data to get your full code coverage metrics. Coverage reports can be generated from your IDE, and they can be exported to Parasoft’s reporting and analytics dashboard DTP. In DTP, users can visually inspect the high-risk areas, understand what should be addressed next, and produce compliance and audit reports.
Why Parasoft Code Coverage Tools?
Parasoft’s solution for code coverage provides critical feedback about the completeness and thoroughness of the testing process, by correlating tests and structural code coverage results. You can use these results or metrics for assessing unit, integration, and system-level testing completeness through our support of all important types of code coverage (function, call, line, statement, block, path, decision, simple condition, and MC/DC), including object/assembly coverage.
Coverage results are available directly in the IDE, with convenient views and highlights in the source code editor, as well as in the form of static HTML or pdf reports, and dynamic reports through Parasoft’s centralized reporting dashboard.
Users can monitor applications executed natively on the desktop, cross-platform using simulators, or on real embedded hardware. C/C++test’s coverage module is optimized to minimize the impact on the execution performance and test binary footprint, which makes it suitable for use with high-end, server-based applications, as well as very limited systems based on 16-bit microcontrollers.
When connected with Parasoft’s Process Intelligence Engine, users benefit from test impact analysis. For each and every test performed, including manual, system-level, or UI-based, tests are recorded for not just test/fail and results but also their coverage impact on the codebase.
Each additional test is overlaid on this existing information, creating a complete picture of test success and coverage. As code is changed, the impact is clearly visible on the underlying record, highlighting coverage tests that now fail or code that is now untested. Raising this information in various degrees of detail allows developers and testers to quickly identify what needs to be altered/fixed for the next test run.
With Parasoft, teams can concentrate on code coverage for the areas of active development, instead of the entire codebase, which can be especially problematic when working with legacy codebases. Rather than solely trying to achieve a coverage number for the entire codebase, Parasoft helps you pinpoint the parts of the code that are changing.
Parasoft’s reporting dashboard correlates the data from C/C++test with observed changes in the codebase to focus the development team on achieving higher levels of code coverage for those specific, modified parts of the codebase. With Parasoft, you can minimize the impact of changes by efficiently managing the change itself.
Frequently Asked Questions
Elevate your software testing with Parasoft solutions.