Agile Software Development: Management and Testing

Policy-Driven Approach

Parasoft Concerto Policy Center converts management expectations into measurable tasks and enforcement points throughout the development process. For example, assume that management wants to implement a policy for developing secure code. Policy Center can turn on the tasks of peer code review, static code analysis, and flow-based static analysis and then parameterize those tasks given specific standards (e.g. PCI DSS, OWASP, HIPAA, or customized rules etc.). Parasoft Concerto then measures and reports to a vast array of management expectations and development tasks, delivering real-time policy compliance data.

Actionable, Measurable Policies

The reactive nature of how management expectations are expressed and measured poses a significant risk to organizations. The lack of a coordinated governance mechanism also severely hampers IT productivity; you cannot improve what you cannot measure.

Parasoft Concerto Policy Center solves this pervasive challenge by translating management goals and expectations into executable and measurable actions. Management expectations are set by defining what practices are required as well as when and how to apply them. Related tasks are then seamlessly integrated throughout the SDLC and unobtrusively monitored for compliance.

For example, policy may require that each "checked in" code modification triggers the appropriate automated tests (static analysis scans, regression test execution, etc.) and peer code review requests–then any reported issues be resolved within two days. Policy compliance is measured and reported in real-time for management visibility. If policy expectations are not met, Parasoft Concerto reminds the responsible team member and ensures that the associated requirement is not marked as complete until the issue is resolved.

Based on Decades of Experience

Parasoft is the industry leader in defect prevention–in fact, we wrote the book on it (Automated Defect Prevention, Wiley-IEEE, 2007). With 20+ years of experience working with 58% of the Fortune 500 companies, Parasoft knows how to seamlessly integrate defect prevention into the SDLC–which is key to making it adoptable and sustainable. Parasoft Policy Center captures this expertise so it can be applied to any development process or environment.

Centralized Policy Application

Parasoft Policy Center can be applied to drive any policy or combination of policies that your organization has decided to enforce. For example:

FDA software validation
PCI DSS
Secure application development
HIPAA
CMMI

[ back to top ]

Centralized Requirements and Issue Tracking

Parasoft's solution is designed to simplify and enhance project planning, process monitoring, and team participation by grouping project requirements, defects/enhancements, and tasks into a single system. It integrates with most bug tracking systems (BTS), requirement management systems (RMS), and Microsoft Project to enhance functionality and make associations between components. In addition, Parasoft ships with its own requirements repository, which also serves as a product backlog. This repository offers backlog-item prioritization and supports a requirement hierarchy.

Release Planning

Parasoft Project Center helps teams plan reasonable release goals for functionality and scheduling expectations. Requirements and issues are split into actionable, measurable work tasks. For each task, the cost and the time required to complete it is estimated by team members.

If a specific deadline must be met, the system helps the team determine what functionality can reasonably be completed by that deadline, given the current resources. When the deadline is more flexible, the system provides on-the-fly assessments of how long it would take to complete different sets of functionality. The team can then work with stakeholders and management to determine a viable course of action.

Iteration Planning

Next, the tasks can be scheduled into iterations or sprints. Using the estimates provided by the team, Concerto helps the team divide the release into iterations by providing on-the-fly calculations of the working time required (and estimated deadline) for different combinations of tasks.

Release Preparation

Before issuing a release, the team needs to assess whether the user stories slated for that release were implemented correctly. Parasoft assists with this at multiple levels. First, it monitors whether the related development and testing tasks are marked as completed. Moreover, it also exposes the results and coverage of tests that validate specific requirements–and it continuously monitors whether the related code meets expectations regarding reliability, security, performance, and compliance to applicable guidelines. This provides objective reassurance that the work is done correctly.

Parasoft also assists with the scheduling, tracking, and documentation of user acceptance tests. These tests can be completed using the manual testing interface, which adds consistency and repeatability to the manual testing process. In addition, teams are alerted if the functionality covered in manual acceptance tests from previous releases might have been impacted by code changes–and those tests need to be rerun.

[ back to top ]

Automated Task Distribution

The planned tasks are automatically distributed to the team members to which they have been assigned (via business rules or via manual assignment). Tasks are accessible via the Concerto Web interface or within the development IDE where developers perform most of their work. This gives developers easy, constant access to an exact plan of attack. They know the overall project goal, their own tasks to complete to help achieve that goal, and the deadline for each task.

Automated Task Monitoring

When a developer is ready to start working on an assigned task, he or she marks it as "in progress," works on the task as normal, then marks it as "done" when it's completed. Without interrupting the developer's workflow, the task distribution technology works behind the scenes to track what files are related to the given task and how much time is spent working on these files. This is important for change-based testing. It also allows the team to compare the actual time spent on tasks versus the expected time allotted for each task, and then update the time estimates for the overall iteration as needed.

Flexibility to Add Tasks and Adjust Estimates

The team member who was assigned a task has the flexibility to submit a new time estimate or create a new task if he or she determines that more work needs to be done. It is the nature of the software development business that initial time estimates for specific tasks are often impossible because there is always hidden work that nobody can foresee... until a team member begins to dig into the code.

In such cases, managers are immediately notified that new work has been added or suggested, and they are given the flexibility to carve out new tasks and distribute the work among the team. This capability allows for better, more accurate information to flow back to the management team.

[ back to top ]

Comprehensive Integrated Testing

The Agile methodology calls for various testing and validation techniques–such as test-driven development (TDD), continuous regression testing, and pair programming–to be continuously applied throughout the SDLC in order to ensure the early and continuous production of working software. Parasoft facilitates these practices with a single comprehensive tool that easily integrates (out-of-the-box) with a wide range of development environments. Automation of complicated and time-consuming quality tasks increases velocity and enables teams to focus on delivering the correct business requirements.

Static Code Analysis

Parasoft's pattern-based static code analysis monitors whether code follows industry-standard or customized rules for ensuring that source code meets uniform expectations around security, reliability, performance, and maintainability. Over 15 years of research and development have gone into optimizing Parasoft's patented static code analysis engine. Although the power of the static code analysis is the combination of analysis engine technology and the strength and flexibility of the rule library, our e ngine uses various static code analysis techniques to optimize the analysis and reports. Our static code analysis solutions feature:

A centralized, integrated system for automated monitoring of code compliance across heterogeneous environments (Java, C/C++, C#, VB.NET, JavaScript, etc.), core industry standards (WS-*, Section 508, FDA, PCI, MISRA, etc.), and organization-specific policies (security, branding, etc.).
Rule sets that are the most comprehensive in the industry and are constantly being extended.
Instant assessments of quality defects as well as security vulnerabilities such as:
- Input-based attacks
- Backdoor vulnerabilities
- Unsafe environment configuration
- Weak security controls
- Deadlocks and race conditions
- Erratic application behavior
- Unsafe error handling and logging
- Exposing sensitive data
Customizable issue prioritization to ensure that the most critical issues are addressed in a timely manner.
Automated refactoring to correct many identified violations.
The ability to graphically define and automatically check custom rules that prevent application-specific errors from reoccurring and monitor adherence to organization-specific policies.

[ back to top ]

Data Flow Static Analysis

Parasoft's data flow static analysis provides automated detection of runtime errors without requiring the software to actually be executed. This enables early and effortless detection of critical runtime errors that might otherwise take weeks to find.

We statically simulate application execution paths which may cross multiple units, components, and files to identify paths that could trigger runtime errors such as:

Java: NullPointerExceptions, resource leaks, accessing arrays out of bounds, unvalidated input in array indexes, incorrect Iterator usage, division by zero, SQL injections, XSS, and other security vulnerabilities.
.NET: NullReferenceExceptions, ArgumentNullExceptions, resource leaks, division by zero, dereferencing before checking for null, SQL injections, XSS, and other security vulnerabilities.
C and C++: Using uninitialized or invalid memory, null pointer dereferencing, array and buffer overflows, division by zero, memory and resource leaks, and dead code.

To simply defect analysis, a complete analyzed path trace for each potential defect is reported in the IDE, and automatic cross-links to code help users quickly jump to any point in the highlighted analysis path.

This ability to expose these errors without executing code is especially valuable for teams with legacy code bases lacking robust test suites or embedded code, where runtime analysis and detection of such errors is not effective or possible.

Code Metrics Analysis

Parasoft calculates various metrics for your code to help you assess your code base and monitor changes. Code metrics calculation identifies brittle or overly-complex code that could impede agility or reuse. It also helps you better understand code complexity and assess the potential impacts of an anticipated code change. This enables you to make more informed decisions as to how to modify, refactor, and test it. In addition to reporting calculations for industry-standard metrics such as Inheritance Depth, Lack Of Cohesion, Cyclomatic Complexity, Nested Blocks Depth, Number Of Children, we enable you customize the acceptable thresholds for each metric, then alert you when metrics are outside of the prescribed range. Leveraging this automation, team resources are freed to focus on analyzing and improving the problematic code- tasks that truly require human intelligence.

Peer Review

Parasoft's automated code analysis relieves developers from having to perform line-by-line inspections during peer reviews, freeing them to focus on higher-level analyses that require human intelligence. With automated checking for compliance to the team's and organization's development policies, the team can begin reviews by discussing interesting findings from the automated code analysis results, then move on to examining design, algorithmic, and implementation issues. This aspect of the peer code review is supported by Parasoft's Peer Review module, which automatically identifies updated code, matches the code with designated reviewers, and tracks the progress of each review item until closure. By automating critical workflow components, we allow the team more time to focus on productive tasks. [ back to top ]

Unit Testing

Parasoft delivers a complete framework to create, manage, and extract greater value from unit tests. We help you exercise and test an incomplete system-enabling you to identify problems when they are least difficult, costly, and time-consuming to fix, which in turn reduces the length and cost of downstream processes such as debugging. Our patented technologies automatically generate tests that not only verify code correctness and reliability at multiple levels, but also capture the code's code behavior to establish a baseline for regression testing. Once this baseline test suite is in place, you can incrementally improve its intelligence and value by:

Extending the automatically-generated xUnit test cases.
Integrating your legacy xUnit test cases (or writing new ones).
Exercising the use case scenarios for the functionality you want to "freeze." Parasoft Tracer will observe the code behavior and automatically design unit test cases with real data that represents the paths taken through the application.

Collectively, these test cases establish a safety net that alerts the team when modifications impact application behavior. Since all tests are written at the unit level, the test suite can be run independent of the complete system to isolate code behavior changes, reduce setup complexities, and make it practical to execute on a daily basis.

For unit testing details tailored to your specific development environment, see the following pages:

Continuous Regression Testing - Unit Level

Parasoft solutions enable continuous regression testing by facilitating the development of a robust regression test suite, as well as fully automating regression test suite execution and assigning each test failure to the responsible developer. A baseline behavioral regression test suite can be generated automatically to capture the project code's current functionality (e.g., with characterization tests). Once this baseline regression test suite is in place, you can incrementally improve its intelligence and value by:

Extending the automatically-generated JUnit, CppUnit, NUnit, or Cactus test cases.
Integrating your legacy JUnit, CppUnit, NUnit, Cactus, or HttpUnit test cases (or writing new ones).
the application can be exercised from the UI or messaging layer) Exercising the use case scenarios for the functionality you want to "freeze." Parasoft Tracer will observe the code behavior and capture it in unit test cases that can be run from a single machine, without accessing the production or staging system.

For unit-level regression testing details tailored to your specific development environment, see the following pages:

Runtime Error Detection

Parasoft's memory error detection automatically identifies a variety of difficult-to-track programming and memory-access errors, along with potential defects and inefficiencies in memory usage. Errors such as memory corruption, memory leaks, access outside of array bounds, invalid pointers, and the like often go undetected during normal testing, only to result in application crashes in the field. We help you find and eliminate such defects in your applications to ensure the integrity of their memory usage. [ back to top ]

Functional Testing

Parasoft's solutions offers the unique ability to create end-to-end test suites that cross application layers, SDLC phases, and testing methods:

With a single test suite, you can continuously validate all critical aspects of complex transactions which may extend through web interfaces, backend services, ESBs, databases, and everything in between.
"Unit tests" can be created as soon as a single "unit of work is completed, then this test suite can be incrementally enhanced to verify additional components as they are added, test the integration of components, and audit end-to-end business processes that span across composite applications.
The same end-to-end test suites that are created for functional testing can also be leveraged for security as well as load testing– and all of these tests can easily be integrated into a robust regression testing suite.
During test execution, you can visualize and trace the intra-process events triggered by tests, facilitating rapid diagnosis of problems directly from the test environment. You can also continuously validate whether critical events continue to satisfy functional expectations as the system evolves.

Parasoft's continuous quality practices promote the reuse of test assets as building blocks to streamline the validation of end-to-end business scenarios impacted by changing business requirements. This enables your team to execute a more complete audit of your business application.

For more details, see the Functional and Load Test Solution page.

Continuous Regression Testing - Application Level

Parasoft's application-level continuous regression testing validates that business processes continuously meet expectations across multiple layers of heterogeneous systems. This reduces the risk of change and enables rapid and agile responses to business demands.

Testing activities, done by both developers and QA, often consume the bulk of project resources. It is essential that resources are optimized to deliver maximum agility without compromising quality.

With Parasoft's solution, you can execute a component-based testing strategy that ultimately allows you to focus on the impact of change. Parasoft's continuous regression tests are applied to the multiple layers throughout your system. These tests will then immediately alert you when modifications impact application behavior providing a safety net that reduces the risk of change, enabling rapid and agile responses to business demands.

You can evolve regression suites for heterogeneous systems rapidly and easily, then leverage the same functional regression test suites-including complex business scenarios and rich message validation assertions-into load tests without scripting.

For more details, see the Functional and Load Test Solution page.

Service Virtualization

Evolving services in a distributed environment and across multiple teams is a complex endeavor due to the interdependencies between the system and business processes. For example, in a system that incorporates multiple endpoints such as credit card processing, billing, shipping, and so on, it may be difficult for one team to test the responses from another team without interrupting normal business transactions.

Parasoft's solution can test complex and distributed environments by automatically generating server stubs based on existing test suites. We can quickly and easily automate server emulation across multiple environments, thereby streamlining the collaborative development and testing activities of multiple teams and ultimately speeding the SDLC.

For more details, see the Functional and Load Test Solution page. [ back to top ]

Load Testing

Parasoft solution's performance testing not only monitors the server's response rate with the specified number and mixture of simultaneous requests, but also verifies whether the test loads cause functionality problems. We facilitate performance testing by allowing you to load test your existing end-to-end functional test suites (which may span from the web interface, through services, to the database) according to preset or customized load test scenarios.

Preset scenarios can be used to verify robustness, scalability and durability. You can easily customize these scenarios to use different test cases, load levels, load distributions, and so on. You can also distribute virtual users across remote server machines to simulate extreme loads and/or test from different locations.

Support is also provided for load testing non-Parasoft components such as JUnits or lightweight socket-based components. This provides teams an integrated solution for their various load testing needs.

For more details, see the Functional and Load Test Solution page.

Security Testing

Parasoft establishes a continuous process that ensures security verification and remediation tasks are not only deployed across every stage of the SDLC, but also ingrained into the team's workflow.

We automate a broad spectrum of security validation and verification practices for C/C++, Java, .NET, SOA, Web, and RIA– the most comprehensive in the industry. Practices supported "out-of-the-box" include:

Static code analysis - Coding standards, data flow, metrics–including preconfigured PCI DSS 6 and OWASP configurations.
Peer code review - Workflow management and automation.
Penetration testing - Message layer and web interface.
Message layer policy validation - Authentication, encryption, and access control.
Runtime analysis - Buffer overflows and other memory errors.
Unit testing - Verification of input validation methods.

For more details, see the Application Security Solution page.

Manual Testing

Parasoft's Web-based manual testing interface can be invoked for any action that truly requires human intelligence or interaction (e.g., user acceptance testing). Developers or testers can define a step-by-step test scenario and then assign this scenario to the related task stored in Parasoft Concerto. Through manual test reports, you can see a high-level, graphical breakdown of each manual testing scenario: what passed, what failed, and the time taken to complete the tests. You can also drill down to see detailed test steps and notes (for example, a particular test failed at step 8 with a segmentation error).

The benefit of performing manual testing through this interface is that each manual step is captured so the complete manual sequence can be easily retrieved, reviewed, and repeated adding consistency and repeatability to a process that traditionally lacks it.

Change-Based Testing

Parasoft's change-based testing helps you optimize your testing efforts by identifying and executing only the test cases directly related to your most recent source code modifications. Not having to test the entire system after each modification yields tremendous productivity improvements.

Requirements-Based Testing

Parasoft's requirements-based testing aligns testing efforts with business needs. In addition to graphically reporting requirement status as indicated by developers, we trace requirements back to test cases. At a glance, the team can gain an objective assessment of which requirements are actually working as expected, and which still require testing. This real-time visibility into true requirement status helps you prevent late-cycle surprises that threaten to derail schedules and budgets.

Test-Driven Development (TDD)

Parasoft supports Test Driven Development with many different capabilities: both automated and assistive. In general, an organization can specify any sequence of development tasks–including writing a test for each requirement–within Parasoft Concerto. The preferred sequences of tasks are defined as "policy" and Parasoft Concerto enforces the desired behavior via a process (or exception-based task notification system). In other words, if developers perform the expected actions (as defined by the policy), then the system remains passive and does not bother them. Notifications are generated only when actions don't align with policy expectations.

[ back to top ]

Tracking Progress

With unobtrusive automated monitoring of work progress, teams can get continuous real-time assessments of how many tasks remain and how much work still needs to be done prior to the deadline. If needed, they can quickly decide whether it is necessary to reschedule tasks to meet the deadline–for example, moving tasks of lower priority to another iteration, or postponing the deadline.

As development completes tasks, QA is continuously updated on development progress and the implementation status of project artifacts (requirements, tasks, defects/enhancements) so they know what's ready for testing.

Throughout the project lifecycle, teams can easily monitor project progress and manage project workflow. A burndown chart shows how the estimate of work remaining has changed over the course of a sprint. A Task Creep chart is also available.

Moreover, teams can continuously optimize their process by monitoring how effectively they are working, what activities are most time-consuming, and where efficiency can be increased.

Analyzing Completed Projects

By tracking and correlating source code, automated tests, and manual tests–as well as tracing them to all project artifacts (requirements, defects/enhancements, and tasks)–Parasoft's solution can provide unparalleled visibility into the development process. This gives the team the essential insight needed to increase velocity and meet scheduling and budget expectations.

Reviewing Historical Development Data

Developers can review historical data, such as the tasks they worked on, along with the source code and tests performed regarding each task. Having the source code monitored is extremely useful for developers. Let's say, for example, it has been some time since functionality was implemented, and it needs to be modified. Or, maybe some bugs need to be fixed. Having historical source code registered in Parasoft Project Center makes future modifications of fixes easier for developers. Parasoft Project Center provides developers with automatic and manual tests, which can be performed on each project artifact (requirements, defects/enhancements, tasks).

Policy-Compliance Reporting

Parasoft Concerto unobtrusively monitors policy compliance and dispatches alerts when additional action is needed to achieve policy compliance.

For example, when functionality needs to be implemented for an application that must be secure (e.g., on online baking system), Concerto can initiate policy-designated tasks such as automatically scanning the code to check compliance to team's designated secure coding policy. It can also prompt team members to perform other policy-designated tasks that require human intervention–such as performing peer code review on security-critical code, writing functional tests that validate security controls, and so on, Whenever possible, policy enforcement tasks are performed "behind the scenes"–with alerts being sent only when policy expectations are not met.

With Concerto handling policy enforcement in this manner, you gain constant and objective visibility into whether the project is being implemented "correctly"–whatever "correctly" means in terms of your specific organization, industry, and project. Essentially, Concerto allows you to implement the appropriate policies in the background, providing you continuous visibility into whether the expected functionality is being implemented on time and on budget–in the context of the policies you put in place.

Requirements and Issue Traceability

Test traceability is realized by specifying a requirement or defect ID, along with a URL for the relevant information, when setting up a test. The status of test execution can then be automatically correlated with requirements or defect status.

Parasoft integrates into the various components of the SDLC and correlates all key artifacts, from tests, to requirements, to code, to project tasks. For instance, each test scenario is automatically linked to a related requirement or task. At any point, you can instantly assess the current level of verification for each requirement or task by back tracing to all associated tests. Moreover, as Concerto monitors the status of code and test artifacts for each requirement, it automatically issues a "Retest Required" alert when it determines that code was modified without rerunning tests.

[ back to top ]