DevOps Best Practices for Automotive Software Development

Use Test Automation to Enable Continuous Integration

It’s important to fully understand the DevOps phases and when to start using the right tools for the best results. There are also various tried and true CI/CD pipeline tools or solutions in the market that support your DevOps deployment. Some tools used by the automotive industry include Parasoft, Jenkins, GitHub, GitLab, Azure DevOps, Bitbucket, Bamboo, Docker (containers), and more.

Continuous Integration

One of the core components in DevOps is continuous integration. CI is the merging of code features, fixes, or small changes backed by process which include version control and software build automation. The goal of CI is to establish a consistent and automated way to build, package, and test applications in what is called a software “pipeline.”

Testing, in most cases, is the roadblock to making CI seamless and efficient. Manual testing is out of the question, so it comes down to automating testing or not testing at all. Automated testing includes more than just unit testing, it also includes best practices such as static analysis and regression testing. Automated testing generates immediate feedback which is used to adjust and replan for the next iteration or sprint.

Test Automation in CI/CD Workflow

Test automation in DevOps involves automating your testing methods. It’s important to be familiar with each of the phases of this methodology and where to trigger your testing verification and validation methods.

You may have also heard the term DevSecOps, which is the practice of integrating security into the DevOps workflow. It requires a mindset shift in teams to integrate security tools and practices into this methodology. Security has become important for many auto manufacturers due to regulatory requirements from WP.29 R155 and the need to incorporate processes from standards like ISO 21434.

An important benefit of the CI/CD pipeline workflow is the feedback loop it provides to make the “continuous” part work properly. Any hangups in the loop cause inefficiencies that lead to delays in development and deployment.

How Test Automation Tools Help With Unit Testing

Test automation tools provide a way around the following unit test hurdles:

• Unit testing and assertion frameworks provide standardized execution formats like CPPUnit.
• IDE integration delivers unit testing results directly to the developer.
• Guided unit test generation reduces the coding effort.
• Function mocking isolates the code from its dependencies.
• Code coverage shows what code was executed.
• Host and target-based test execution.

By making testing more efficient within the CI/CD pipeline, it’s possible to improve test coverage with each cycle and not impede the progress of the project with continuous testing.

Remove Validation & Verification Roadblocks With Continuous Testing on Host & Target Systems

Continuous integration is just part of a continuous development process that needs testing and delivery to reap the benefits of the approach.

Continuous testing provides an automated, unobtrusive way to obtain immediate feedback on a software release candidate. Continuous testing isn’t simply more test automation. The purpose is to build quality and security into the product as part of a continuous integration/release/delivery process.

Some Continuous Testing Activities

• Use static analysis early for detection of bugs and security vulnerabilities. Early detection, usually at the developer’s desktop, prevents bugs from wasting unit testing time and entering the software build.
• Enforce a coding standard to conform to required corporate and industry standards, like MISRA C/C++ or SEI CERT C. Adhering to a coding standard prevents whole classes of defects and poor coding practices from entering the build to become larger issues later on.
• Automate test execution on host and target systems as soon as units are coded and when integrated subsystems are ready. The required tests that need to verify units also include nonfunctional, load, security, and performance testing. These tests are executed directly from the CI orchestration system. The results from these tests get pulled back into the same build and gathered. Code coverage information (statement, branch, and MC/DC) is cross referenced by unit, file, test, and build number.
• Maintain requirements traceability to correlate code, tests, and other assets with stakeholder requirements. This provides an objective assessment of the requirements that are working as expected, which ones require validation, and the ones at risk.
• Use test impact analysis to focus where testing efforts need to go. From a risk perspective, changed code impacts more than the software itself. It impacts relevant tests and assets. As teams make code changes, questions arise: Do we need new tests or modify existing ones? What are the impacts on dependencies? Automation helps teams focus only on the tests that are impacted.
• Maintain test data automatically to increase the effectiveness of a continuous testing strategy. Good test data and test data management practices increase coverage and drive more accurate results. However, developing or accessing test data can be a considerable challenge in terms of time, effort, and compliance.
• Rely on test and data generation to make continuous testing work. You can continuously generate data appropriate for the type of scenario you’re trying to execute instead of trying to rely on production data sources and hoping that all the right data is in the right place. Combining data generation with simulation will allow you to inject the right data in the right place at the right time.

Continuous Testing on Host & Embedded Automotive Target Systems

At the development level, automotive software isn’t much different than typical application development. It requires IDEs, compilers, static and dynamic analysis, and build tools. However, tools often target different architectures than they work on, for example, host versus target environment. Versions of tools are important to ensure a homogenous development environment across the team.

Automating testing for automotive software is more challenging due to the complexity of initiating and observing tests on embedded targets, not to mention the limited access to target hardware that software teams have. Software test automation is essential to making automotive testing workable on a continuous basis from the host development system to the target system.

Testing automotive software is particularly time consuming. Automating the regression test suite provides significant time and cost savings. In addition, test results and code coverage data collection from the target system are essential for validation and standards compliance.

Traceability between test cases, test results, source code, and requirements must be recorded and maintained, which means data collection is critical in test execution.

A solution like Parasoft C/C++test comes with an optimized test harness to take minimal additional overhead for the binary footprint and provides it in the form of source code, where it can be customized if platform-specific modifications are required.

Leverage Containers to Create Consistent, Reproducible, & Secure Development Environments

Development teams know that CI/CD workflows can function the same, regardless of whether they’re containerized or not. The value that containers provide DevOps teams is that it allows applications to be easily deployed and patched. Organizations can scale containers to their needs to accelerate development, testing, and production within Agile and DevOps use cases.

When it comes to managing complex development environments, especially in the safety-critical space, teams usually struggle with the following challenges:

• Synchronizing upgrades for the entire team to a new version of a tool like a compiler, build toolchain, and so on.
• Dynamically reacting to a new security patch for the library or software development kit (SDK).
• Assuring consistency of the toolchain for all team members and the automated infrastructure or CI/CD.
• Ability to version the development environment and restore it to service the older version of the product that was certified with a specific toolchain.
• Onboarding and setting up new developers.

All these problems are easy to solve with containers. Within each container is the standardized versions of the development toolchain including static analysis tools, compiler, IDE, and build tools. These containers are updated in a centralized fashion and deployed on an as-needed basis when a developer is working on project code.

The consistency of the development environment eliminates any errors that might arise from developers having slightly different versions of tools on their desktop. It’s a good way to ensure everyone is using the correct version of each tool, which might be important later for regulatory compliance. Containers also provide a centralized way to update tools and ensure these updates are propagated immediately across the organization.