Continuous Integration (“CI”) is a well-understood and (at this point) well-adopted practice. It is a necessary first step to significantly improve the velocity of application delivery.
Continuous Integration allows developers to push their changes into a “master” branch of source code, with a single developer potentially pushing many changes to the master branch throughout a single day. To ensure that the master branch is pristine, buildable, and high-quality, it is critical to test after every change, since it serves as the golden copy of source code for the application.
(If you’re interested in more about continuous integration, I recommend Martin Fowler’s older but still interesting article here on the history of integration in software development and the benefits/best practices of CI.)
Today we’re going to focus on how to configure Parasoft SOAtest to execute functional regression testing as part of a Continuous Integration process. In this article, I’ll describe the steps to configure SOAtest with Jenkins, a popular automation platform. We’ll use the open-source Parabank application and deploy it using Docker to keep things simple.
The diagram below explains what we’ll be setting up in this article. It’s best read from left to right.
In short, Jenkins will check out a repo from Github which contains the SOAtest project called “Parabank” that contains REST tests. Jenkins will also pull a Docker image called parasoft/parabank from Docker Hub. This image contains not only Parabank, but also Tomcat and the correct Java runtime environment.
Jenkins will then run an instance of this Parabank image (called a “container”). Afterwards, Jenkins will tell SOAtest to run the tests we pulled from Github so that we can validate our Parabank instance.
Now, this isn’t quite in the spirit of true continuous integration (since I’m giving you a pre-built application), but I wanted to use Docker to save you the trouble of having to build Parabank with Maven and having to install and configure Tomcat/Java.
A more realistic/real-world diagram of CI is provided below. The Developer checks source code into Github. Now we want to test that the application is still in a good state even after the Developer’s changes.
The source code change in Github triggers a build in Jenkins, and Jenkins kicks off an automated Maven build (which executes JUnit tests). If all unit tests pass, then the packaged application (parabank.war) is deployed onto Tomcat. SOAtest then begins to execute functional “black box” tests.
Only after the unit tests pass (during the Maven build) and the functional “black box” tests pass (during SOAtest execution) are the developer’s original changes considered good.
Let’s get to the steps necessary to configure the process in the first diagram!
soatestcli.exe -fail -data %TEMP_WORKSPACE_PATH% -resource /Parabank -config “builtin://Demo Configuration” -localsettings %LOCALSETTINGS_PATH%
If you open the test in the SOAtest Desktop UI, you’ll find out that this failure is primarily a test data/test environment configuration issue. Our loan processor declined a loan that it should have approved.
Test environment configuration and test data are the biggest obstacles to reliable automated testing. In future articles, I’ll explore how a technology called service virtualization can help ensure that we always have the exact environment configuration we need in order to run our tests reliably at any time – this will segue us from Continuous Integration into the world of Continuous Testing.