RELIABLE & REPEATABLE TESTS
ON-DEMAND AUTOMATED TEST CASES
Alaska Airlines is primarily a West Coast carrier that services the states of Alaska and Hawaii with mid-continent and destinations in Canada and Mexico. Alaska Airlines received J.D. Powers’ “Highest in Customer Satisfaction Among Traditional Carriers” recognition for twelve years in a row even recently winning first in all but one of the seven categories.
A large part of the credit belongs to their software testing team. Their industry-leading, proactive approach to disrupting the traditional software testing process ensures that testers can test faster, earlier, and more completely.
Learn how Ryan Papineau and his team used advanced automation in concert with service virtualization to rigorously test their complex flight operations manager software. The result: operations that run smoothly— even if they encounter a snowstorm in July.
Watch a preview of the Alaska Airlines presentation at the recent Automated Software Testing & Quality Summit. Learn how they used virtualization to efficiently scale testing for their complex flight operations application.
At Alaska Airlines, the flight operations manager software is ultimately responsible for transporting 46 million customers to 115 global destinations via approximately 440,000 flights per year, safely and efficiently. This software coordinates a highly complex set of inputs from systems around the organization to ensure flights are on time while evaluating and managing fuel, cargo, baggage, and passenger requirements.
In addition to the previously mentioned requirements, the system considers many factors including weather, aircraft characteristics, market, and fuel costs. It’s highly complex with numerous dependencies (see Figure 1) and functions as the heart of a well-run airline.
The complexity and huge number of essential services as dependencies made it nearly impossible for Alaska Airlines to build repeatable tests. In addition, the test environment was shared with other teams and constantly changing. The changes impacted the flight management software and required persistent tracking and debugging. The ongoing resolution of these issues hindered their own development progress.
Data management was another large issue. It entailed various databases throughout the system updating at different intervals, making test data management a huge challenge. The various services in the system were another key challenge. It was nearly impossible to simulate real world events in a running production system.
Given the need to decouple their application under test from a very complex and constantly evolving environment, Alaska Airlines looked to service virtualization as a solution to their testing woes.
They use Parasoft Virtualize to decouple their application under test from the myriad of dependencies in their complex test environment. Alaska Airlines started using service virtualization as a proxy between their client devices and the airport web services. They ran through various scenarios to capture the requests and responses.
These captured scenarios became the basis of repeatable, concise, and precise tests for later use. However, this was just one aspect of their system operation.
The full application isolation required more virtualization and test data management. Alaska Airline’s virtualization environment included replacing all key systems with their virtualized equivalents as shown in the Figure 3.
Test data management was key to solving the issue with inconsistent databases in their real-world environment. Alaska Airlines decoupled the application from the dependent real-world databases by using Parasoft Virtualize test data management capabilities. They used IBM Optim in conjunction with custom T-SQL data transforms against data captured using Parasoft Virtualize.
Alaska Airlines captured enough data for a full day of real operations. Then deployed it within the test environment to create a “Groundhog Day” set of scenarios — repeating the same events and responses for a complete 24 hour cycle, progressing the time while keeping the scenarios consistent.
Alaska Airlines’ solution for their passenger services is a point of pride. Using service virtualization, they were able to simulate passengers going through the complete lifecycle —booking, checking in, boarding — for all types of flights in their system.
The pilot’s interaction with dispatch and fuelers is only a couple of the interactions they have with the system flight management system. It’s nearly impossible to coordinate these interactions in real time without impacting production resources and personnel.
The team simulated these complex interactions with dispatch and fuelers. Getting consistent results, and no longer needed to leverage critical personnel to operate the airline.
Alaska Airlines freed themselves from the confines of testing in a complex and volatile real-world environment — to test the untestable. Once their flight management system test environment was decoupled from its many dependencies, they reaped the benefits of service virtualization.
After adoption was complete, they became sophisticated users. Alaska Airlines increased their test capability to include cloning test data, fabricating scenarios, and creating tests where scenarios ran continually, vastly scaling the testability of their application under test.
“If you can dream it, you can build it. We essentially went from being untestable to testable to now having a thriving demand for more of these capabilities.”
—Ryan Papineau, senior software engineer at Alaska Airlines
They have 500 automated test cases that leverage this virtualized test environment. Those test cases run as needed, improving their automated execution by magnitudes over conventional wait-and-see or hunt-and-peck testing models.
Thanks to all this simulation and automation, testers can test whatever scenario they want, whenever they want. Their tests are now 100% reliable and repeatable. They eliminated all the variables that previously caused false positives and wasted time. Enabling testers to instantly access the exact test environment configurations that their test plans call for helps Alaska Airlines ensure smooth flight operations when faced with anything from a brief air traffic control outage to the improbable July snowstorm.
Another key learning experience and benefit for Alaska Airlines was a cultural change that made the organization more open to try new things. Now, the bigger the problem to solve, the easier the adoption of new techniques and tools is for the organization. The benefits that the Parasoft service virtualization solution offered their team include the following.
Decoupling from the complex production environment, which allowed Alaska Airlines to isolate their application and generate the test cases and environment that would accelerate their testing.
Repeatable, concise, precise test scenarios were invaluable to Alaska Airlines and enabled them to scale the types and frequency of their testing. The team expanded and customized scenarios as needed and injected specific scenarios for special test cases. The repeatability of these tests sped up the development cycles and the consistency removed a serious challenge they had with the real-world operation environment.
Parallel testing environments reduced the wait time impact. The ability to clone and customize scenarios allowed Alaska Airlines to speed up their simulated test cycle. Parallelization sped up testing but also meant specific time-dependent scenarios occurred in shorter time frames.
Test data management enabled the team to decouple production databases and ensure data consistency, which plagued Alaska Airlines’ operating environment. They reused test data captured from real-world systems to create life-like test scenarios that were difficult to coordinate in real time. Parasoft Virtualize test data management allowed the team to replicate databases consistently and in minutes. Something difficult to achieve in their real environment.
Find the ideal service virtualization starting point for your team and identify the best deployment model for your requirements. Download the whitepaper.