How a Medical Device Co. Cut Testing Costs in Half

Bovie Medical Corp. is a leading maker of medical devices and supplies as well as the developer of J-Plasma®, a patented new plasma-based surgical product. Bovie Medical is also a leader in the manufacture of a range of electrosurgical products and technologies. The Company also leverages its expertise through original equipment manufacturing (OEM) agreements with other medical device manufacturers.

Recently, Bovie Medical decided to introduce a generator into the market that hospitals could interact with via a software user interface. The interface is an LCD screen that enables doctors to view pertinent patient medical information while performing surgery.

By adopting an automated embedded testing solution in house, Bovie Medical was able to move verification and validation (V&V) testing in-house for their medical devices with embedded software—cutting these costs approximately in half.

Weighing Options: Testing Vendor or In-house V&V

Part of meeting FDA requirements and regulations includes performing clinical trials and providing documented proof that their medical devices are safe for patients. Before clinical trials can begin, Bovie Medical performs extensive software testing.

Previously, Bovie Medical turned to a vendor for their embedded software testing. When it came time to begin development of their latest project, management presented the Software Development and Quality Assurance (QA)  departments with a few choices for testing.

Option 1—Returning to the Testing Vendor

The first option was to return to the vendor and have V&V performed outside Bovie Medical. However, the development team saw several drawbacks to this choice. Although the vendor did a good job of breaking down the 45 to 50 thousand lines of code and analyzing it, they did not find all of the bugs.

Being a general testing company, the vendor did have the advantage of automated tools to perform static analysis, code reviews, and even system testing. However, this was outweighed by a significant disadvantage: the vendor lacked understanding of the products being tested.

The Bovie Medical development team knows their system, what they’re developing, and what the end user will do with it. They know the industry and are best equipped to test what they develop themselves. Gary Malfa, Software Engineer at Bovie Medical, specifies some other disadvantages of returning to the vendor. “We would have to give them all of our proprietary source code.

“Plus, we would still have to spend a lot of time with a company like that—explaining how to interact with our software from a user point of view. They would have to understand how our software works before they could test it. Based on the previous project they did for us, handling all of their questions was like a full-time customer support job.”

Of course, the greatest concern was the cost. When this vendor tested Bovie’s previous release, it cost $300,000. Going forward with this choice would mean going back to the same company and paying them near that amount again to perform another round of V&V.

Option 2—Working with Testing Consultants

A second option was to hire a couple of independent consultants to come in and perform the V&V. The challenge here was finding qualified people to do the job. Bovie Medical could not seem to find a consultant who was a true expert in V&V.

Beyond that challenge, this option entailed putting heavy resources toward training the consultants as they simultaneously performed the V&V work at hand— a major drawback.

Option 3—Implementing Automated Testing In-house

The last option combined hiring another QA tester with purchasing automated testing software so that the development team could perform all V&V in-house. From a capital expenditure point of view, this option was the most cost-effective of the three options. In fact, according to Gary Pickett, CFO, going this route would save Bovie Medical roughly 50% of its validation costs for the release at hand.

The Bovie QA department found support and encouragement to pursue this option after they attended an FDA seminar. A team member asked, “Is it admissible for a developer to test his own code using automated testing tools for static and dynamic testing?” The representative replied, “We think automated testing is on the cutting edge of software validation in the medical arena. We are encouraging people to do that.” With that, the QA and development teams concluded that it was worth searching for a way to bring the software validation in-house and cut costs.

Meeting FDA Guidelines

Bovie Medical’s machines do not hook up to an internal network within the hospital. The machines are not client server applications. As mentioned earlier, Bovie Medical’s medical devices are standalone and use embedded software programs.

Doctors use these embedded software medical devices in their operating rooms. They look at the screen to view essential medical information of the patient on whom they are performing surgery. Because the medical devices require specialized embedded software programs, Bovie Medical uses a custom compiler.

Bovie Medical developers load a test executable based on the actual C++ embedded project directly onto the embedded target device, and then run unit tests directly on the hardware that’s going to be running in the hospital environment. Data is fed into each individual unit. After unit tests are finished running, the results are retrieved and uploaded to the developer’s PC into their testing solution.

All of that data, along with documentation, is stored in Bovie Medical’s testing archive. In the event that they get audited by the FDA, that vital information is just a click away.

Finding and Removing Hidden Logic Issues—Early

Their embedded testing solution enables Bovie Medical developers to run static analysis on their source code before moving onto dynamic testing. The static analysis verifies that all established language rules are not violated.

Malfa says, “It teaches our development team better coding habits so that we can make our source code more fault-tolerant, more robust.” Beyond that, it enables Bovie Medical’s development team to find bugs and design flaws before projects go into production.

Malfa explains, “I’m not talking about obvious bugs that software developers can find without automated testing tools; I’m talking about difficult-to-find logic issues that can take many hours of manual unit and integration testing to discover.”

Malfa discusses a memory leak that their embedded testing solution found in his code. He says, “This was the type of bug that could not be found by doing all of the human testing in the world. The software could be released and run fine for ten years. Then, one day out of the blue, it would run in a hospital with a certain sequence and certain data, then fail.”

He goes on to say that his team could investigate an error like that for months and never be able to reproduce it. ”

Getting to Market Sooner

Malfa is emphatic that their embedded testing solution has saved Bovie Medical an immense amount of time. “If we had to manually do code reviews and manually run all of our unit tests, we’d have to hire and train a dedicated staff of people to get it done in the same amount of time.” He also talks about the alternative of going back to that testing vendor. He states, “We are able to get our product to market approximately 6 months sooner with an automated embedded testing solution than we could have if we had gone back to the testing vendor.”

Leaping Forward with Automation

In closing, Malfa states, “Developing software without using an automated software testing solution that finds bugs and design flaws during the development cycle is like developing software in the dark ages. Not only is it expensive, but it also increases the chances of embarrassment by a customer finding a bug. Now that we’re automating embedded testing, we find and fix software issues in the development stage long before the QA department starts testing the code. Developing software products with an automated testing solution process in place is a quantum leap forward for the software industry.”

By Parasoft

Parasoft’s industry-leading automated software testing tools support the entire software development process, from when the developer writes the first line of code all the way through unit and functional testing, to performance and security testing, leveraging simulated test environments along the way.