Recently, I was reading a post on LinkedIn in which someone had asked the difference between several static analysis security vendors. One person, unsurprisingly a vendor, replied that their solution was better because while other companies focused on quality and security, they strictly do security.
Of course, that was a ridiculous statement. And perhaps this kind of thinking is indicative of the current rampant problem with application security in the industry; for instance, organizations that are attempting to run their security group completely separate from the rest of the SDLC (both development and testing efforts). In this model, the security team runs their own tests, mostly attempting to break the software, then feeds security bugs back to the dev team. In other words, attempting to test security into their code. I can assure you this is no more effective than testing quality into your code.
Sure, this kind of security testing is necessary, but it simply is not enough. While breaking the software is indeed useful, reliance on it as a method of improving security leads to errors being found at a point that is too late, and they end up being suppressed. In particular, root-cause issues such as improper frameworks and algorithms are swept under the rug, as schedule wins the conflict between rewriting the code and getting the release out the door.
In the Linkedin comment I mentioned above, the vendor is dangerously misleading an unsuspecting prospective customer by claiming their software is somehow better, without actually saying anything useful about how or why it’s better. I don’t mean to pick on any particular tool vendor, especially since I work for one. However, I’m frustrated by such strawman arguments, that give the appearance of hawking snake-oil. In this case, the vendor’s product may indeed have interesting unique features, but we’re left with the impression that security is somehow magically different than quality, which lowers our understanding of application security and makes us all a little less safe.
Security has to be treated like quality, and quality has to be based on mature engineering practices, because the truth is that if you have a quality problem, you have a security problem. Studies show that anywhere from 50-70% of security defects are quality defects. In other words, good old-fashioned quality bugs are turning about to be vulnerabilities that intruders/hackers/bad actors use to penetrate your application (we call those “zero days”).
“The consensus of researchers is that at least half, and maybe as many as 70% of common software vulnerabilities are fundamental code quality problems that could be prevented by writing better software. Sloppy coding.”
– Jim Bird “Building Real Software”
If you still aren’t sure how quality and secure overlap, take a look at a couple of examples from the CWE Top 25. The following possible security outcomes are from the CWE Technical Impact work:
If you go further into the full CWE list (over 800 items), you find many others, i.e. all forms of overflow/underflow, initialization, uncontrolled recursion, etc. These are all common security attacks, as well as obvious quality issues.
The complexity of software systems grows very rapidly. Trying to test every possible variation quickly becomes nearly impossible. As Richard Bender puts it, “The number of potential tests exceeds the number of molecules in the universe,” which is just a more fun way of saying you’ll never get it done. Or from Jim Bird, “for a big system, you would need an infinite number of pen testers on an infinite number of keyboards working for an infinite number of hours to maybe find all of the bugs.”
So both security and reliability have to be designed and engineered in. You can’t test them in. As long as security is something “extra” it will suffer.
Here are a few things you can do to start improving your software quality and security at the same time.
So we need to start building security into our code. This is best way to really harden it, rather than just patch known holes. Having all your software development results from coding, building, and testing integrated into a central repository provides control, measurement, and traceability. This is the basis for future improvement.
And remember, the cost of solid prevention is less than the cost of dealing with bad or insecure software. So there’s really no excuse.
Arthur has been involved in software security and test automation at Parasoft for over 25 years, helping research new methods and techniques (including 5 patents) while helping clients improve their software practices.