DO 178B/C Compliance with Parasoft

DO 178B/C / ED 12B/C Software Development and Testing with Parasoft

DO-178B/C (referred to as ED 12B/C in Europe) provides guidance for aerospace software engineers so they can ensure the airworthiness of the systems they develop. The FAA’s airworthiness requirements determine whether or not the agency will issue their approval document called a TSO, or technical standard order. The standard is not explicitly mandated, but it is a critical component of the TSO process, thus a requirement for gaining FAA approval.

Those with experience developing embedded software systems that comply with industry standards will recognize the qualifying language used in DO-178B/C. RTCA, the not-for-profit consortium that wrote the standard, specifically states that the organization’s recommendations "represents a consensus of the aviation community" in regard to software development. In this way, DO-178B/C is descriptive, rather than prescriptive. The standard describes the goals of the processes, rather than prescribing the methods by which they are achieved.

DO-178C introduces 5 Software Levels (SLs) A to E. Software Level for a system is determined during safety assessment process based upon the failure condition which may result from the anomalous behavior of the software. Software Level A represents the most stringent level, which anomalous behavior can effect in catastrophic failure condition for the aircraft, what follows, level A introduces the most rigorous practices to the software development and testing.

DO-178C defines objectives, related activities and output for the software development and testing processes which depend on the Software Level classification. For example, teams developing systems with SL A will have to document that MC/DC Source coverage was collected during the software testing process to the sufficient level. Teams developing systems with SL B, C or D do not need to worry about MC/DC coverage but may need to worry about statement or branch Source coverage.

To comply with DO-178C, organizations need to implement multiple processes and requirements which are typically supported with software tools. Usage of the tools in DO-178C processes is regulated and has to be preceded with analysis and determination of the Tool Qualification Level. Tool Qualification Levels span from 1 to 5 where 1 is the most rigorous and 5 is the least rigorous. Details of qualification, which is a formal process of approving tool for use. Details of the qualification process are described in the accompanying standard DO-330.

In most respects, DO-178B and DO-178C are identical, but some of the broader goals, processes, and definitions have been explained in greater detail in DO-178C. For example, in chapter 6.1, which defines the purpose for the software verification process, DO-178C adds the following purpose with regard to the Executable Object Code:

  • The Executable Object Code is robust with respect to the software requirements such that it can respond correctly to abnormal inputs and conditions.

This is in addition to the statement regarding the verification of the Executable Object Code defined in DO-178B:

  • The Executable Object Code satisfies the software requirements (that is, intended function), and provides confidence in the absence of unintended functionality.

The additional requirement further defines the role of the Executable Object Code to ensure the safe functionality of the system, whereas the previous minimum scope of the verification process may lead to system failure. There are many instances where the language in DO-178C seeks to define concepts, functions, and process in more detail, but an exhaustive list of such updates are outside the scope of this paper.

How to Leverage Parasoft Tools for DO 178B/C Compliance

Manually driving policy throughout the development process isn’t feasible in most organizations. Parasoft’s solution for automating software verification and validation processes and software quality tasks specified in DO-178B/C, including static analysis, data flow static analysis, metrics analysis, unit testing and runtime error detection, provides teams a practical way to prevent, expose, and correct errors in functional safety systems.

The following tables matches key principles from DO-178B/C with Parasoft capabilities. The tables are intended to provide a sample of Parasoft functionality and is by no means complete.

5.3.2: Software Coding Process Activities

Method Parasoft C/C++test capability

The Source Code should implement the low- level requirements and conform to the software architecture.

The Source Code should conform to the Software Code Standards.

The Source Code should be traceable to the Design Description.

Inadequate or incorrect inputs detected during the software coding process should be provided to the software requirements process, software design

Static analysis with Coding Standards Compliance.

Rules that enforce industry best practices are built in.

Users can select entire rule sets based on industry standards, individual rules, or create custom rules based on the organization's policies

6.3.3: Reviews and Analyses of the Software Architecture

Method Parasoft C/C++test capability

The objective of these reviews and analyses is to detect and report errors that may have been introduced during the development of the software architecture. These reviews and analyses confirm that the software architecture satisfies these objectives:

  • Compatibility with the high-level requirements: The objective is to ensure that the software architecture does not conflict with the high-level requirements, especially functions that ensure system integrity, for example, partitioning schemes.
  • Consistency: The objective is to ensure that a correct relationship exists between the components of Configure Parasoft rules to enforce any kind of policy, including a policy that specifies software architecture and a policy based on target computer specifications. Parasoft includes software development testing tools, such as static analysis, unit testing, code coverage analysis, peer review analysis, and runtime error detection to verify that the code performs as expected. Rules that enforce industry best practices are built in. Users can select entire libraries based on industry standards, individual rules, or create custom rules based on the organization's policies.the software architecture. This relationship exists via data flow and control flow.
  • Compatibility with the target computer: The objective is to ensure that no conflicts exist, especially initialization, asynchronous operation, synchronization and interrupts, between the software architecture and the hardware/software features of the target computer.
  • Verifiability: The objective is to ensure that the software architecture can be verified, for example, there are no unbounded recursive algorithms.
  • Conformance to standards: The objective is to ensure that the Software Design Standards were followed during the software design process and that deviations to the standards are justified, especially complexity restrictions and design constructs that would not comply with the system safety objectives.
  • Partitioning integrity: The objective is to ensure that partitioning breaches are prevented or isolated.

Configure Parasoft rules to enforce any kind of policy, including a policy that specifies software architecture and a policy based on target computer specifications.

Parasoft includes software development testing tools, such as static analysis, unit testing, code coverage analysis, and runtime error detection to verify that the code perform as expected.

Rules that enforce industry best practice are built in. Users can select entire libraries based on industry standards, individual rules, or create custom rules based on the organization's policies

6.3.4: Reviews and Analyses of the Source Code

Method Parasoft C/C++test capability

Source Code complies with low-level requirements

Source Code complies with software architecture

Source Code is verifiable, Source Code conforms to standards

Source Code traceable to low-level requirements.

Source code is accurate and consistent

Users can create test cases to verify correctness of low level requirements implementation. Test cases can be annotated with requirement ID for full traceability reporting, which presents testing results in context of requirements.

Users can configure rules to enforce any kind of policy, including a policy based on software architecture.

Parasoft includes software development testing tools, such as static analysis, unit testing, code coverage analysis, peer review analysis, and runtime error detection to verify that the code performs as expected.

Rules that enforce industry best practices are built into Parasoft C/C++test. Users can select entire libraries based on industry standards, individual rules, or create custom rules based on the organization's policies.

Pattern-based data flow analysis checks all possible paths simulates test case execution and detects errors across multiple units, components, and files to ensure that the code is accurate and consistent.

6.4.1: Test Environment

Method Parasoft C/C++test capability

More than one test environment may be needed to satisfy the objectives for software testing. An excellent test environment includes the software loaded into the target computer and tested in a high fidelity simulation of the target computer environment.

Note: In many cases, the requirements based coverage and structural coverage can be achieved only with more precise control and monitoring of the test inputs and code execution than generally possible in a fully integrated environment. Such testing may need to be performed on a small software component that is functionally isolated from other software components.

Certification credit may be given for testing done using a target computer emulator or a host computer simulator. Guidance for the test environment includes:

  • Selected tests should be performed in the integrated target computer environment, since some errors are only detected in this environment.

Parasoft C/C++test has a number of test environment capabilities:

  • Tests can be executed on the target device, target processor emulator/simulator or on the host computer.
  • Full support for a wide variety of embedded RTOSes and architectures is provided. Tests can be cross-compiled together with generated test harness and code instrumentation for a desired target runtime environment.
  • Full customization of the test execution sequence is also supported. In addition to using built-in test automation, users can incorporate custom test scripts and shell commands to fit the tool into their specific build and test environment.

6.4.3: Requirements-based Testing Methods

Method Parasoft C/C++test capability

Requirements-Based Hardware/Software Integration Testing:

Typical errors revealed by this testing method include:

  • Incorrect interrupt handling.
  • Failure to satisfy execution time requirements.
  • Incorrect software response to hardware transients or hardware failures, for example, start-up sequencing, transient input loads and input power transients.
  • Data bus and other resource contention problems, for example, memory mapping.
  • Inability of built-in test to detect failures.
  • Errors in hardware/software interfaces
  • Incorrect behavior of feedback loops.
  • Incorrect control of memory management hardware or other hardware devices under software control.
  • Stack overflow.
  • Incorrect operation of mechanism(s) used to confirm the correctness and compatibility of field-loadable software.
  • Violations of software partitioning.

Requirements-Based Software Integration Testing:

Typical errors revealed by this testing method include:

  • Incorrect initialization of variables and constants.
  • Parameter passing errors.
  • Data corruption, especially global data.
  • Inadequate end-to-end numerical resolution.
  • Incorrect sequencing of events and operations.

Requirements-Based Low-Level Testing:

Typical errors revealed by this testing method include:

  • Failure of an algorithm to satisfy a software requirement.
  • Incorrect loop operations.
  • Incorrect logic decisions.
  • Failure to process correctly legitimate combinations of input conditions.
  • Incorrect responses to missing or corrupted input data.
  • Incorrect handling of exceptions, such as arithmetic faults or violations of array limits.
  • Incorrect computation sequence.
  • Inadequate algorithm precision, accuracy or performance

Parasoft’s C/C++test unit testing framework allows creating advanced tests scenarios which can be supported with stubbing/mocking framework for tested code isolation and simulation of complex failure conditions. Precise test scenarios help in detecting coding errors, logical problems and requirements discrepancies.

Stub analysis and generation is facilitated by the Stub View, which presents all functions used in the code and allows users to create stubs for any functions not available in the test scope or to alter existing functions for specific test purposes.

The advanced interprocedural static analysis module simulates feasible application execution paths and determines whether these paths could trigger specific categories of runtime bugs. Defects detected include:

  • Using uninitialized or invalid memory
  • Null pointer dereferencing
  • Array and buffer overflows
  • Division by zero
  • Memory and resource leaks
  • Dead code

6.4.4.2: Structural Coverage Analysis

Method Parasoft C/C++test capability

Structural Coverage Analysis: The objective of this analysis is to determine which code structure was not exercised by the requirements-based test procedures. The requirements-based test cases may not have completely exercised the code structure, so structural coverage analysis is performed and additional verification produced to provide structural coverage.

Guidance includes:

  • The analysis should confirm the degree of structural coverage appropriate to the software level.
  • The structural coverage analysis may be performed on the Source Code unless the software level is A and the compiler generates object code that is not directly traceable to Source Code statements. Then, additional verification should be performed on the object code to establish the correctness of such generated code sequences. A compiler generated array-bound check in the object code is an example of object code that is not directly traceable to the Source Code.
  • The analysis should confirm the data coupling and control coupling between the code components

A multi-metric test coverage analyzer, including statement, branch, path, and MC/DC coverage, helps users gauge the efficacy and completeness of the tests, as well as demonstrate compliance with test and validation requirements.

Back tracing from coverage elements to the corresponding test cases enables users to analyze test results and extend the test cases for better coverage, with higher efficiency. All test results can be exported for compliance purposes.

Object Coverage module (aka Assembly Coverage) helps users to determine if there is any object code generated by the compiler which is not traceable to the source code and ensure that it was examined during testing procedures.

6.4.4.3: Structural Coverage Analysis Resolution

Method Parasoft C/C++test capability

Structural Coverage Analysis Resolution: Structural coverage analysis may reveal code structure that was not exercised during testing. Resolution would require additional software verification process activity. This unexecuted code structure may be the result of:

  • Shortcomings in requirements based test cases or procedures: The test cases should be supplemented or test procedures changed to provide the missing coverage. The method(s) used to perform the requirements- based coverage analysis may need to be reviewed.
  • Inadequacies in software requirements: The software requirements should be modified and additional test cases developed and test procedures executed.
  • Dead code: The code should be removed and an analysis performed to assess the effect and the need for re-verification.
  • Deactivated code: For deactivated code, which is not intended to be executed in any configuration used within an aircraft or engine, a combination of analysis and testing should show that the means by which such code could be inadvertently executed are prevented, isolated, or eliminated. For deactivated code, which is only executed in certain configurations of the target computer environment, the operational configuration needed for normal execution of this code should be established and additional test cases and test procedures developed to satisfy the required coverage objectives

Parasoft C/C++test provides powerful requirements traceability reporting. Traceability reports present testing results, including unit test cases execution results and code coverage results in context of appropriate requirements. Analysis of requirements traceability reports allows users to detect several classes of problems including:

  • Incomplete requirements
  • Requirements inaccuracies
  • Requirements implementation problems
  • Dead code

Parasoft’s C/C++test unit testing framework allows creating advanced tests scenarios which thoroughly and precisely exercise low level requirements. Tests scenarios can be supported with stubbing/mocking framework for tested code isolation and simulation of complex failure conditions. Precise test scenarios help in detecting coding errors, logical problems and requirements discrepancies.

12.2: Tool Qualification

Method Parasoft C/C++test capability

Qualification of a tool is needed when processes of this document are eliminated, reduced or automated by the use of a software tool without its output being verified as specified in section 6. The use of software tools to automate activities of the software life cycle processes can help satisfy system safety objectives insofar as they can enforce conformance with software development standards and use automatic checks.

Only deterministic tools may be qualified, that is, tools which produce the same output for the same input data when operating in the same environment. The tool qualification process may be applied either to a single tool or to a collection of tools.

The software verification process objectives for software development tools are described in paragraph 12.2.1, item d. A tool may be qualified only for use on a specific system where the intention to use the tool is stated in the Plan for Software Aspects of Certification. Use of the tool for other systems may need further qualification.

C/C++test fully supports the qualification process as described in DO-178C/DO-330.

C/C++test is supported with Qualification Kit, which automates the tool qualification process, and guides users through all the steps required to conduct the qualification and produce necessary documentation.

C/C++test’s Qualification Kit automates significant part of the tool qualification process, reducing the amount of manual work and eliminating this distraction from software teams. Dedicated Qualification Support Tool guides users through all the steps required to qualify the tool and automates majority of tedious manual work required to qualify the tool including executing the test cases from the exhaustive test suite provided together with the Qualification Kit and generation of final reports required to document the qualification process.

Developing DO-178B/C-compliant software for airborne systems is no easy feat. But Parasoft helps ease the burden by offering a broad range of development testing and verification tools, including coding standards compliance analysis, data and control flow analysis, unit testing, application monitoring, workflow components. When paired with a policy-driven development approach that bridges that gap between development activities and business processes, Parasoft C/C++test becomes a powerful tool for enforcing compliance with DO-178B/C and other standards. Development teams can also generate configurable test reports that facilitate the traceability requirements for the software verification process. The testing functions, configurable contexts and reporting mechanisms in Parasoft C/C++test arms software development teams with proven tools that help them achieve DO-178B/C compliance.