To reduce the risks associated with their business-critical transactions, a leading New Zealand bank and financial-services provider wanted to extend their existing GUI-based testing to more extensively exercise the application logic within internal systems.
This logic resides within their ESB: a message broker component that features 80+ services and 750+ operations. The ESB represents a single point of failure within the infrastructure—but the company was unable to test it directly or automatically.
With the expertise of Parasoft partner IntegrationQA and Parasoft’s market-leading API Testing solution, the company gained a service testing methodology supported by automated, reusable tests that performed a broad scope of testing directly from the service/API layer.
The result: they saved over $2.1 million NZD in testing and development costs over an 18 month period.
The company’s existing functional testing process provided some degree of black-box testing for the integration points between front-end and back-end systems. However, the company feared that this approach of relying on the front-end as the sole entry point for testing new functionality, technology updates, and large-scale changes to the existing infrastructure was not deep and rigorous enough to ensure that the application logic continuously satisfied the company’s high reliability standards.
To reduce their risk exposure, they wanted to extend their existing end-to-end test scenarios to include direct and extensive testing of the ESB and associated services. However, they lacked access to the domain expertise and technology required to take their existing testing process to this level.
To help the company achieve their quality objectives, IntegrationQA worked with them to establish a dedicated test team who would specialise in testing the infrastructure at the service level using Parasoft API Testing solution. Recognizing the importance of implementing new technologies within an effective process, IntegrationQA also helped them develop a new testing methodology which included:
Using the new technologies and methodologies, the team was able to “shift left” their functional testing from the System Testing phase to the Unit and Integration Testing phases. By exposing more defects when it’s easier, faster, and less expensive to fix them, they achieved significant time and cost savings.
Across all projects, 65% – 100% of the defects uncovered by the Service Test team were identified during the Unit and Integration Testing phases. Approximately 75% of the defects originated at the ESB level, which received limited test coverage with their previous testing methods (as underscored by the fact that one major ESB defect was identified for every 10,000 records processed). Across all projects, between 55% and 100% of discovered defects were classified as Severity 2 and could not be deferred.
Cost Reduction per Test Environment
Once confidence in the Service Test team grew and the new service testing methodology added proven value to the company’s quality efforts, it became an integral part of all pipeline project test planning and estimations. To accommodate the increased workload, the team grew in size from 3 initial members to 10 full-time technical test analysts. IntegrationQA took on the role of team lead, consultant, and facilitator for multiple internal projects at various lifecycle stages.
As the new processes and technologies became more ingrained and mature, the team expanded the scope of work to include performance testing of the ESB, mobile application message testing, ETL testing of the data warehouse, as well as application testing of internal monitoring engines for customer accounts and transactions.
The company has experienced the following benefits from using Parasoft API Testing to implement the recommended service testing methodology:
Over 18 months, the company has saved over $2.1 million NZD in testing and development costs (based on the company’s internal defect cost valuations).
Read the 12-page API Testing technical paper to get the nitty-gritty details about how this company established a service testing methodology supported by automated, reusable tests that perform a broad scope of testing directly from the service/API layer.
You’ll get a close look at how they were able to create reusable, easily-maintainable tests that could robustly exercise any of the available ESB services via 3 main test patterns:
If you’re considering bringing an API Testing tool into your own organization, don’t miss Forrester’s new Functional Test Automation Wave report, which compares the top 9 API testing tools.
The process for developing the current Wave included generating a detailed taxonomy of 40 product attributes, gathering and validating data for nine vendors through lab-based evaluations and client interviews, then placing the evaluated vendors in one of four categories: Leaders, Strong Contenders, Contenders, and Risky Bets.
The report emphasizes that API testing is a critical component for modern application delivery: “In order to support continuous quality and faster development, especially for those on an Agile and DevOps journey, testing must shift from a UI-only approach to a UI-and-API approach.”
Want to learn about the fundamentals of API Testing? Take a look at the API Testing Wikipedia page.
Parasoft’s industry-leading automated software testing tools support the entire software development process, from when the developer writes the first line of code all the way through unit and functional testing, to performance and security testing, leveraging simulated test environments along the way.