Get immediate feedback about code changes using test impact analysis

The quicker you can test, the quicker you can release. But you don’t have to wait for the nightly/daily build to run your full suite of unit tests for validation about the impact of your code changes. Instead, get immediate insights into the tests impacted by your code changes, and be more confident at check-in.

Unit testing has been an industry best practice for years, and as development teams build out their test suites, they grow larger and larger. And as the tests expand into integration or component level tests, they take longer to run. With new trends in unit testing, such as TDD, these test suites will grow even larger than before, since all code depends on tests, and more of them.

Having a large foundation of unit tests can be a great foundation for testing, but this can come with a big impact on test execution time, especially as the unit tests expand into integration/component level tests. So the key to knowing what to test is in understanding the exact impact each code change has, which tests need to be run, and what new tests may be required.

Calculating code coverage is one aspect of determining what’s been tested, but by itself it’s not enough. The real magic happens when code coverage is calculated for each test and then correlated with changes in the code on a build-by-build basis. Using this approach gives us test impact analysis – which is the process by which we identify exactly which test to execute to validate code changes. By leveraging test impact analysis for unit testing with Parasoft Jtest, software development teams can focus their testing efforts and truly accelerate their development pipeline, via the IDE or CI process.

Test impact analysis at your fingertips

Finding and fixing bugs as early as possible is the primary benefit to having test impact analysis.  With results of test impact analysis integrated directly into the IDE, developers can seamlessly leverage its benefits within their workflow, to focus testing efforts exactly in the right place and ensure that code changes are completely tested, including indirect impacts on related code.

Although finding and fixing bugs early is the primary goal, there are other advantages to having the results of test impact analysis at the developer’s fingertips, including:

• Highlighting tests that actually execute code within the modified files, ensuring that only required tests are run.
• Integrating with a variety of source control and build systems with the ability to be customized to support a unique environment when needed.
• Identify not just direct dependencies between tests and code, but indirect dependencies as well, to ensure the full impact of changes is covered by testing.
• Easy initial configuration and the ability to automatically identify and run the affected tests without the user needing to do anything but change code and view the results.

When combining test impact analysis with the CI process, time savings are evident and you can focus the development team’s effort on exactly what needs to be done to ensure quality.  Optimizations at the development and build times are critical in achieving agile objectives when dealing with changes.

Focused testing means better software, delivered on time

Finding bugs sooner is better and cheaper than finding them later in the software development lifecycle, which can cause major schedule slippages. Developers often don’t know which tests to run, so they either don’t run any tests, or run too few. In this case, they depend on the build, which is set up to run the entire suite of tests, so development teams are idle as they wait for feedback/validation from the build process regarding their changes. By instead leveraging test impact analysis, development teams can find and run the appropriate tests before even committing code into a build to validate the changes.

Test impact analysis also means that developers can get faster feedback on code changes that cause test failures from their CI process. Development managers want their teams to run tests before the code is checked in, ideally, but it’s often not done. In addition, they want to ensure that their teams know as soon as possible, after code is checked in, if the code broke any tests. Therefore, it’s important for the test impact analysis capability to span the CI process as well as the developer’s desktop.

Parasoft Jtest’s “Impacted Unit Tests” view in the IDE

So what does this look like in practice? In the IDE, as the developer is writing code, Jtest’s “Impacted Unit Tests” view provides a (dynamically evolving) list of which tests need to be run to exercise the code that is being locally changed. Then, all the developer needs to do is simply right click on the impacted test and execute it or simply run all impacted tests.

Jtest keeps track of the affected tests that have been run, and clearly displays which have passed and which have failed, making it easy for the developer to determine which tests still need to be run, or which tests have failed and need to be addressed. Once all tests have been run and are passing, the developer has more confidence in their code change and can safely commit their code and move on to the next development task.

The code-test-feedback-remediate workflow is specifically designed to increase productivity at the developer’s desktop, allowing users to easily identify and run just the tests that are needed to verify local code changes. In a deploy-directly-to-production model, this prevents bugs from making it to end users. When multiple teams are involved in a project, this prevents time lost to other teams needing to trouble-shoot and resolve problems that could have been avoided had the appropriate tests been run before the code was committed.

Integrating and Complementing the Pipeline Tools

Tools don’t work if you can’t easily integrate them into your existing toolchain. Parasoft Jtest supports projects that are in Git or SVN source control, as well as other source control systems. The IDE integration exists for both Eclipse and IntelliJ. Jtest provides Maven and Gradle plugins that can be integrated into a CI build job that is running tests as part of a Maven or Gradle build.

These plugins can be configured to fully automate a test suite by determining which code has changed since the baseline build, which is often configured to be the last nightly build, and then determining which tests need to be run to exercise that code, and then run just that subset of tests. This allows a team to set up a CI job that only runs tests based on the most recent code changes, allowing them to shrink the amount of time it takes to run a CI job potentially from hours to minutes.

As a best practice, teams can run the full suite of unit tests on a nightly basis and leverage test impact analysis in the daily (interim) builds. This makes Parasoft Jtest especially beneficial for teams with long-running test suites, who can get true CI by getting results from relevant tests within minutes after committing code. Without being able to do this, bad code changes can introduce regressions that are not caught as quickly and thus can sneak into production or interfere with work being done by other team members.

By focusing resources on exactly what needs to be tested, development teams can quickly and efficiently run tests to verify the code they are checking in on a constant basis – catching bugs and security vulnerabilities before they ever reach the build.

By Kapil Bhandari

Kapil is a Product Manager at Parasoft, focusing on Parasoft Jtest. Kapil held several technical positions ranging from software engineer to development lead, before moving into product management.