Compliance in software development may mean slightly different things to different organizations. The differences in attaining compliant software likely relate to the marketplace where they sell their products. They range from rigorous development for safety-critical software to enterprise applications that require security and privacy capabilities for handling sensitive data.
Standards and regulations emerge in different industries and are put in place to protect customers and users from harm. That harm could manifest itself as physical danger or financial loss of credentials and sensitive personal information. Often these standards are a reaction to events that resulted in injury or major data breaches.
In terms of safety, there are several standards some of which include:
There are also standards for security and privacy such as GDPR for general data protection/privacy and PCI-DSS for secure handling of credit card information.
For most standards, compliance demonstrates in various ways that the software you’re deploying meets the requirements of the standard. The development process produces documentation, reports, and other artifacts that show compliance. It keeps audits in mind and can include demonstrations and acceptance testing.
The amount of time and cost for compliance depends on the level of rigor the industry standard requires. These standards define the design and development guidelines to follow based on the safety or security impact that the software has on the end user. Failure of the product can lead to death or injury in severe cases. For example, ISO 26262 defined for levels of safety integrity level (A, B, C, D), where ASIL A is the lowest safety risk or impact, and D the highest.
The probability of occurring plus the severity of the risk and controllability determines where a product fits on the safety integrity level. (Controllability is the ability of the driver to mitigate the risk.):
The following table shows the severity, exposure, and controllability translates to the various ASIL in ISO 26262:
Figure 2: Hazard Analysis and Risk Assessment
EN 50128 uses the terms software safety integrity levels (SSIL) with similar criteria. Both the ISO 26262 and EN50128 standards derive their definitions from IEC 61508, which is an overarching standard for functional safety.
These SSIL levels are also based on probability of failure and the severity of the failure on life and limb. As expected, the level or rigor increases as the ASIL/SIL level requires and compliance costs rise with it. The level of scrutiny increases with risk. As such, there’s an impact on software development cost and schedule, which development teams can improve with automation.
The biggest risk to software organizations is software failure that can cause death, injury, or other type of harm that opens the gates to liability. To help cut this risk, software organizations should comply with safety and security standards. Organizations must demonstrate that they made all reasonable engineering choices to ensure the safety and security of the user/customer/operator. Creating safer and secure software requires a commitment to the spirit and the letter of industry standards. 1
Discovering significant safety and security issues late in the software lifecycle requires significant rework, retesting, and re-documenting. It’s expensive and takes time to resolve. It also points to a possible failure in the system and engineering process.
It’s critical to shift left compliance to as early as possible in the software development lifecycle (SDLC). (We recommend shifting left for all quality, security, and safety practices.) To this end, our recommendations include the right tools, process, and automation to help reduce software risk and increase compliance without negatively impacting cost and schedule.
Frequently, the adoption of culture and process that suits the application space is lacking in many organizations. Safety-critical software, for example, requires a high level of rigor in software development that implies higher quality throughout.
Organizations aligning to safety and security requirements carry a heavy workload that involves providing an audit trail of compliance for each software stage and process. Along with usual software development, the additional work includes:
In most cases, coding standards are applicable, for example MISRA for ISO 26262. So teams must work to correct deviations from the standards and measure compliance.
Functional tests are necessary to prove that products meet requirements. This is in addition to safety and security requirements tests. All the tests require traceability and audit documentation.
Combined, the increased level of traceability, testing, strict development standards, and documentation add significantly to development costs products. It makes the risk of compliance issues downstream even more daunting.
The key to reducing the workload and tedium of these processes is automation.
Software development tools help reduce the workload and errors of the compliance processes by automating the most repetitive and tedious processes. In addition, tools are better at keeping track of the project history and relating results against requirements, software components, tests, and recorded deviations.
Tools also help enforce good practices such as coding standards, unit testing, code coverage, traceability, and more. Teams can adopt tools early in the lifecycle to help shift left the compliance workload.
The following sections discuss how Parasoft tools help reduce the risks and costs of compliance.
Developing safety critical software is no easy feat. Parasoft helps ease the burden by offering a broad range of development testing and verification tools. Here are some of those tools:
The standards that Parasoft directly supports include:
Software development teams can achieve compliance and save time with Parasoft C/C++test. The proven tool provides testing functions, configurable contexts, and reporting mechanisms that assist with advanced software testing methodologies.
Teams have a practical way to prevent, expose, and correct errors in their software with automatically generated reports and dashboards. They can prepare the documentation required for audits with the help of automated tool qualification.
Parasoft helps organizations perform static analysis, unit testing, structural code coverage, requirements traceability, and other testing methods recommended by the standard.
Parasoft’s Assembly Coverage Tool (ASMTool) satisfies the executable object code recommendations set out by DO-178B/C Level A. With little effort, ASMTool can generate structural coverage reports from executable object code.
Structural coverage is reported with easily reviewable insight into compiler-generated code that is not directly traceable to source code statements. It also tracks switch statements, providing full insight into the test execution paths at the machine language instruction level.
ASMTool also supports collecting structural coverage from both the software integration process (target hardware testing) and desktop development environment (Parasoft C/C++test for unit testing).
Coding standards play an important role in many safety and security standards. In most cases, the only practical way to enforce compliance with a coding standard like AUTOSAR C++14 (used in automotive systems) is with a static analysis tool. Parasoft C/C++test supports mapped checkers for the coding standards, so developers don’t need to dereference warnings back to the standards.
Teams can customize Parasoft’s interactive reporting system to the requirements imposed by whichever rules they use (AUTOSAR C++ 14, HIC++, MISRA, and the like) and enable an efficient daily workflow. Developers can check the compliance of their code without leaving their IDEs and integrate the scanning process into CI builds on servers.
Parasoft also supports security-focused standards such as SEI CERT. It helps organizations detect security vulnerabilities in their code and manage the process of achieving compliance with the standard. The high accuracy of the code checkers, low level of false alarms, and sophisticated compliance reporting consistent with the CERT standard help minimize related costs and overhead.
Parasoft’s unique compliance reporting provides a dynamic view of the development process. Interactive reports and customized dashboards provide different views into the results of CERT compliance status and use the wording and categorizations that are defined by the standard, to make it easier to understand.
Combining several test automation techniques results in a lot of quality information. This data is useful, but the sheer volume makes it difficult to gain insight into the quality of your deliverables and make decisions about the release or changes in the development process.
To support decision making, it’s beneficial to aggregate the data into a centralized view of quality. Graphical dashboards provide a top-level view of testing progress. Correlating quality data to your requirements and user stories provides true bidirectional traceability so you can assess the business risk and impact of gaps in your testing practices.
Parasoft’s reporting and analytics dashboard aggregates data from across all the different testing practices including static analysis (like AUTOSAR or MISRA coding standard compliance), code coverage and traceability from unit testing, API testing, and system-level functional validation. This provides a centralized view of quality that’s a real-time audit or your process and visibility into the status of compliance with the required standard. This analysis also generates the documentation needed to demonstrate compliance with an auditor.
Just as examples, let’s look at reporting from Parasoft C/C++test and Parasoft DTP for MISRA and SEI CERT.
Parasoft C/C++test provides dedicated reporting for documenting compliance to MISRA C. A dashboard on the Parasoft web portal provides at-a-glance views on the current state of the project, such as the one here:
Each of these dashboard widgets is linkable to a more detailed view, containing detailed violation reports, files, and source code.
From here, you can automatically create the reports needed to document MISRA compliance as outlined in MISRA Compliance 2016: Achieving Compliance With MISRA Coding Guidelines. Automating these reports is a big time-saver, greatly reducing the amount of manual work required to document project compliance.
Although the SEI CERT C standard doesn’t require specific compliance reports, it does require a project to document conformance to the rulesets (such as L1, L2, and fully compliant.) Parasoft C/C++test includes a dedicated dashboard for CERT C conformance, that looks like this:
Team leads can use this dashboard view to dig deeper into specific areas of concern and assign tasks to developers to increase compliance over time. Viewing the results in context of the risk assessment framework used by the coding standard itself (for instance, seeing specific violations of L1 guidelines), significantly streamlines the process. Automating this reporting reduces the amount of analysis that team leads and architects need to perform in order to achieve CERT C compliance.
Tool qualification is a mandated process for safety-critical software development. Parasoft’s Qualification Kits for Parasoft C/C++test automate the process of creating the supporting documentation required for tool qualification of static analysis, unit testing, and coverage requirements, reducing both the potential for human error and time taken to perform tool qualification.
Reducing the risk and cost of tool qualification includes the following:
Automatically generate compliance documentation. Parasoft automates the creation of documentation required for qualifying Parasoft C/C++test for use in safety-critical industries by walking the user through an intuitive workflow and generating documentation that’s critical for demonstrating tool qualification.
Reduce the scope of what’s needed to qualify the tool. The first step in Parasoft’s efficient workflow is selecting the specific use cases and capabilities of Parasoft C/C++test being used in the software project, to reduce the overall scope of what needs to be qualified and streamline the qualification process.
Automatically execute tests. Although the process of tool qualification cannot be 100% automated, the unique workflow of Parasoft’s qualification kits makes it as painless as possible, reducing manual testing efforts and executing automated tests for selected use cases.
Given the role of Parasoft products at each stage of development is outlined, how does Parasoft reduce risk and save money in software compliance?
Multiple testing technologies integrated into one tool. The comprehensiveness of Parasoft C/C++test translates into increased developer efficiency when developing applications with functional safety requirements. Developers can focus on their core activities without having to learn, integrate, and qualify several tools. Tool integrators or architects don’t have to spend time implementing interfaces between tools for exchanging data and generating uniform reports. Instead, they get all of it from Parasoft out of the box.
A proven test suite designed specifically for functional safety-critical applications. When performing unit testing or runtime memory monitoring, all components required to build the test binary, including the test cases for tested code stimulation and stubs, are expressed in the form of source code and can be versioned and inspected. This approach is superior to other tools in which the stimulation for tested code is sent at runtime from the host and requires additional conversions before assigning to the variables in memory. C/C++test removes this unnecessary layer and assures that the state of the memory before executing tests is constructed in the same way as in the production systems.
Eliminate overhead for functional safety compliance. Parasoft provides team-wide data integration, convenient reporting customized to different coding standards, and advanced analytics through Parasoft’s award-winning Process Intelligence Engine. Users benefit from the ability to aggregate information across multiple different sources (such as source code or requirements management systems, testing tools, or ALMs) for unique data analytics, helping focus efforts in the most productive way. Teams can increase productivity by monitoring trends in the development process, and easily generate reports that comply with industry standards such as “MISRA Compliance: 2016”.
Building safety and security-critical software require rigorous processes that are time-consuming and expensive to implement and for good reason: the failure of such software can have drastic consequences. Industry standards are put into place to help guide companies to build high-quality software to ensure safety and security in developed products. However, meeting these industry standards is costly. That cost lies in both the development process and the artifacts required to document the engineering process. Organizations can reduce costs with the right applications of tools and modern software techniques.
Parasoft provides the right tools and support for processes that help reduce the risk and costs of software compliance with a proven, unified tool suite that reduces the “friction” caused by using multiple tools from multiple vendors. An integrated solution also greatly simplifies tool qualification, a critical aspect of compliance.
Having an integrated platform allows for unique data analysis collected from multiple tools at various stages of development to focus resources on the most important goals. This centralized view of compliance, quality, and security is key to making sound decisions and greatly reducing the guesswork in achieving compliance.
Arthur has been involved in software security and test automation at Parasoft for over 25 years, helping research new methods and techniques (including 5 patents) while helping clients improve their software practices.