Parasoft Logo
Blog LanguageArrow
User Name:
Password:
Sign Up
  Home > Capabilities > Load Testing
Functional Audit – End-to-End Testing
Nucleus

Parasoft's continuous quality practices promote the reuse of test assets as building blocks to streamline the validation of end-to-end business scenarios impacted by changing business requirements. This enables your team to execute a more complete audit of your business application.

Benefit: Reduces the risk of business downtime and ensures business continuity.

Arrow Left Menu - Slider Arrow Right

Parasoft solutions facilitate software defect prevention best practices to help organizations ensure that all software development activities meet uniform expectations around security, reliability, performance and maintainability. With Parasoft solutions, these software defect prevention best practices have minimal impact on the team's workflow because an automated infrastructure handles key software defect prevention tasks in the background. Source code is pulled from source control, then analyzed automatically; developers are notified only when evaluation or response is needed.

Parasoft solutions support and automate numerous best practices for software defect prevention. These software defect prevention best practices can be divided into two major categories: code analysis and code review.

Code analysis includes static (rules-based) analysis, flow analysis, and metrics. This code analysis typically relieves developers from having to perform line-by-line inspections during peer code reviews. Instead, code reviews can begin by discussing interesting findings from the automated code analysis results, then move on to examining design, algorithmic, and implementation issues. This aspect of the peer code review is supported by Parasoft's Code Review module. It automatically identifies updated code by scanning the source control system, matches the code with designated reviewers, and tracks the progress of each review item until closure.

The most critical component to delivering quality as a continuous process is to establish organization-wide quality policies that can be objectively defined, realistically applied, and automatically monitored for compliance. Parasoft can help you establish attainable and monitorable quality policies, including:

  • Development rules for code security, reliability, performance, and maintainability.
  • Workflow rules within your defined process.
  • Quality milestones to measure process efficiency and effectiveness.
  • Quality metrics to measure compliance to the organization's quality expectations.

Policy Management

Parasoft provides centralized management of organization-level and team-level policies for ensuring application security, reliability, performance, and maintainability. Policies are easily deployed and configured for specific projects without compromising the integrity of the corporate objectives. The policies are centralized for simple access and automated for rapid scanning and reporting, including:

  • A rule library with thousands of configurable rules, including specific (contextual) security rules.
  • A graphical RuleWizard module that makes it very simple to construct and maintain customized rules.
  • Configuration/parameterization to customize policies to the unique demands of individual projects.

Workflow Management

Parasoft defines, automates, and monitors a workflow that improves development productivity and forms the foundation for a sustainable quality process. As quality tasks are defined and automated, process control guidelines are established to measure process progress as well as software quality. With monitorable quality gates and thresholds throughout the SDLC, your organization and team can track progress and ultimately improve the software development process.

Workflow Optimization

Tasks to support quality policies must be optimized so they can become an integral part of the team's existing workflow. The lack of automation, repeatability, or consistency will degrade any quality initiative that the organization intends to deploy. Parasoft optimizes the workflow to ensure that the quality process is both sustainable and scalable. For example:

  • Routing each reported issue directly to the responsible developer and customizable issue prioritization both ensure that your most critical issues are addressed in a timely manner.
  • Centralized configuration management to ensure that test configurations are applied consistently and can be updated effortlessly as priorities and processes evolve.
  • Monitoring manual processes that require human intelligence, then converting them into automated processes so manual efforts do not need to be repeated.

Parasoft's code analysis monitors whether code meets uniform expectations around security, reliability, performance, and maintainability. We provide a foundation for producing solid code by exposing structural errors and preventing entire classes of errors. Our code analysis includes pattern-based (rules-based) analysis, data flow analysis, and code metric calculation. All are centrally managed and highly automated. An automated framework is provided to ensure consistency across development languages, development teams, and third-party partners.

Pattern Based Analysis

Parasoft's pattern-based code analysis monitors whether code follows industry-standard or customized rules for ensuring that code meets uniform expectations around security, reliability, performance, and maintainability. Over 15 years of research and development have gone into optimizing Parasoft's patented pattern-based analysis engine. Although the power of the pattern-based analysis is the combination of analysis engine technology and the strength and flexibility of the rule library, our engine uses various code analysis techniques to optimize the analysis and reports. Our solutions feature:

  • A centralized, integrated system for automated monitoring of code compliance across heterogeneous environments (Java, C/C++, C#, VB.NET, JavaScript, etc.), core industry standards (WS-*, Section 508, MISRA, etc.), and organization-specific policies (security, branding, etc.).
  • Fast, easy ways to define and check custom rules that prevent application-specific errors from reoccurring and monitor adherence to organization-specific policies.
  • Customizable issue prioritization to ensure that the most critical issues are addressed in a timely manner.
  • Automated refactoring to correct many identified violations.

Data Flow Analysis

Parasoft's data flow analysis provides automated detection of runtime errors without requiring the software to actually be executed. This enables early and effortless detection of critical runtime errors that might otherwise take weeks to find. We statically simulate application execution paths–which may cross multiple units, components, and files–to identify paths that could trigger runtime errors such as resource leaks, SQL injections, and other security vulnerabilities. This ability to expose these errors without executing code is especially valuable for teams with legacy code bases lacking robust test suites or embedded code (where runtime analysis and detection of such errors is not effective or possible).

Code Metrics

Parasoft calculates various metrics for your code to help you assess your code base and monitor changes. Code metrics calculation identifies brittle or overly-complex code that could impede agility or reuse. It also helps you better understand code complexity and assess the potential impacts of an anticipated code change. This enables you to make more informed decisions as to how to modify, refactor, and test it. In addition to reporting calculations for industry-standard metrics such as Inheritance Depth, Lack Of Cohesion, Cyclomatic Complexity, Nested Blocks Depth, Number Of Children, we enable you customize the acceptable thresholds for each metric, then alert you when metrics are outside of the prescribed range. Leveraging this automation, team resources are freed to focus on analyzing and improving the problematic code– tasks that truly require human intelligence.

Parasoft's automated code analysis relieves developers from having to perform line-by-line inspections during peer code reviews, freeing them to focus on higher-level analyses that require human intelligence. With automated checking for compliance to the team's and organization's development policies, the team can begin code reviews by discussing interesting findings from the automated code analysis results, then move on to examining design, algorithmic, and implementation issues. This aspect of the peer code review is supported by Parasoft's Code Review module, which automatically identifies updated code, matches the code with designated reviewers, and tracks the progress of each review item until closure. By automating critical workflow components, we allow the team more time to focus on productive tasks.

Automated Code Review

Parasoft's automated code analysis checks code as it is written to prevent entire classes of errors and ensure that code meets uniform expectations around security, reliability, performance, and maintainability. This typically relieves developers from having to perform line-by-line inspections during peer code reviews, freeing them to focus on higher-level analyses that require human intelligence. With automated checking for compliance to the team's and organization's coding policies, the team can begin peer code review by discussing interesting findings from the automated code analysis results, then move on to examining design, algorithmic, and implementation issues.

Peer Code Review

Parasoft automates and manages the peer code review workflow. Our Code Review module, which automates preparation, notification, and tracking of peer code reviews, addresses the known shortcomings of this very powerful inspection method. It automatically identifies updated code by scanning the source control system (if the organization reviews code after source control check in) or the local file system (if code is reviewed prior to check in), matches the code with designated reviewers, and tracks the progress of each review item until closure. With the Code Review module, teams can establish a bulletproof review process-where all new code gets reviewed and all identified issues are resolved.

Parasoft delivers a complete framework to create, manage, and extract greater value from unit tests. We help you exercise and test an incomplete system-enabling you to identify problems when they are least difficult, costly, and time-consuming to fix, which in turn reduces the length and cost of downstream processes such as debugging. Our patented technologies automatically generate tests that not only verify code correctness and reliability at multiple levels, but also capture the code's code behavior to establish a baseline for regression testing. Once this baseline test suite is in place, you can incrementally improve its intelligence and value by:

  • Extending the automatically-generated xUnit test cases.
  • Integrating your legacy xUnit test cases (or writing new ones).
  • Exercising the use case scenarios for the functionality you want to "freeze." Parasoft Tracer will observe the code behavior and automatically design unit test cases with real data that represents the paths taken through the application.

Collectively, these test cases establish a safety net that alerts the team when modifications impact application behavior. Since all tests are written at the unit level, the test suite can be run independent of the complete system to isolate code behavior changes, reduce setup complexities, and make it practical to execute on a daily basis.

Baseline

If your team has a large code base with minimal tests or no tests, the fastest route to regression testing is to automatically generate a regression test suite that captures the code's current behavior– and then run it daily to determine if code modifications break existing functionality. To help you achieve this, we scan the project code base, then automatically generate a baseline set of unit test cases that capture the code's current behavior. Then, we establish a continuous regression testing process that ensures code modification impacts are identified and addressed daily, and the test suite stays in synch with the evolving application. This provides a safety net that helps developers rapidly change code with confidence, which is especially critical for teams working on complex and constantly-evolving applications.

Extended

Parasoft provides a variety of technologies to help you extend the automatically-generated test cases to verify specific functionality requirements and/or increase coverage. We provide a number of ways to help you increase the value of the automatically-generated test suite with minimal effort; for example:

  • Manually extending the generated xUnit (JUnit, CppUnit, NUnit, Cactus, HttpUnit, etc.) test classes.
  • Using an object repository and flexible stubs framework to make tests more realistic.
  • Leveraging test data stored in a data source to extend the coverage of existing tests.

Manual

Parasoft solutions can execute your existing xUnit (JUnit, CppUnit, NUnit, Cactus, HttpUnit, etc.) test cases, allowing you to centralize your test execution, reporting, and coverage analysis. Our reporting and authorship calculation capabilities help the team track which test cases failed, since when, and who is responsible for fixing each failure. The team can run all of their tests in command-line mode each night. In the morning, each developer can import the regression failures caused by their modifications; this provides instant feedback on whether their code changes broke the existing functionality. Since each regression failures are directed to the developer responsible for them, the overall process of fixing them is much more streamlined than it would be if all developers had to review the same long list of test failures.

Functional

Parasoft's Tracer technology provides a fast and easy way to create the realistic test cases required for functional testing. Simply use the application's UI or Parasoft's SOA/Web solution to execute the use cases you want to verify. Tracer automatically designs unit test cases with real data that represents the paths taken through the application. No coding or scripting is required. If the functionality associated with your "traced" use cases later breaks, you will be alerted by the failure of the related test cases.

The generated unit tests directly correlate tests to source code, which improves error identification and diagnosis, and allows developers to run these test cases without having to depend on access to the production environment or set up a complex staging environment. This facilitates collaboration between QA and Development: QA can provide developers traced test sessions with code-level test results, and these tests help developers identify, understand, and resolve the problematic code.

Runtime error detection exposes critical defects that occur as the application is exercised. This is key for efficiently identifying defects that manifest themselves only at runtime and zeroing in on the root causes of the application crashing, running slowly, or behaving unpredictably.

C and C++ Memory Error Detection
For C and C++, errors such as memory corruption, memory leaks, access outside of array bounds, invalid pointers, and the like often go undetected during normal testing, only to result in application crashes in the field. Parasoft Insure++ will help you find and eliminate such defects in your applications to ensure the integrity of their memory usage. During testing, Parasoft checks all types of memory references, including those to static (global), stack, and shared memory - both in user's code and in third party libraries. Errors detected include:

  • Corrupted heap and stack memory
  • Use of uninitialized variables and objects
  • Array and string bounds errors on heap and stack
  • Use of dangling, NULL, and uninitialized pointers
  • All types of memory allocation and free errors or mismatches
  • All types of memory leaks
  • Type mismatches in global declarations, pointers, and function calls
  • Some varieties of dead code (compile-time)

C Lightweight Runtime Error Detection
For C, Parasoft C++test can also perform memory error detection at the application level or during unit test execution. We detect C-related memory errors such as memory leaks, null pointers, uninitialized memory, buffer overflows, and more. The instrumentation used to perform this runtime error detection is lightweight and suitable for running on the target board, simulator, or host for embedded testing. The collected problems are presented directly in the developer's IDE with the details required to understand and fix the problem (including memory block size, array index, allocation/deallocation stack trace etc.). Coverage metrics are tracked to help the team measure and increase the scope of their testing efforts.

Java Runtime Error Detection
For Java, Parasoft Jtest can perform runtime error detection on complete applications (e.g. as manual or scripted tests are performed) or while it executes JUnit tests. As the application executes, we analyze it–applying a configurable set of dynamic "runtime rules" that verify the application?s runtime behavior. The rule violations reported indicate errors that actually occurred during execution. Thus, defects are reported with unprecedented accuracy. Categories of errors detected include:

  • Application crashes
  • Eclipse development
  • Exceptions
  • Functional Errors
  • File I/O
  • Graphical User Interface
  • Database
  • Network
  • Optimization
  • Portability
  • Security
  • Servlets
  • Threads & Synchronization

Coverage metrics are tracked to help the team measure and increase the scope of their testing efforts.

Web, SOA, Cloud Runtime Error Detection
For Java-based web, cloud, and SOA applications, Parasoft SOAtest performs runtime error detection as functional tests or penetration tests execute. We correlate each reported error with the functional test that was being run when the error was detected. This correlation between violations and functional tests allows you to trace each reported error to particular use cases against your application.

Parasoft's SOA solution assists you in managing the complex and distributed nature of SOA. Given that your SOA is more than likely to span multiple applications, departments, organizations and business partners, this requires a component-based testing strategy. With the Parasoft solution set, you can execute a component-based testing strategy that ultimately allows you to focus on the impact of change. Parasoft's continuous regression tests are applied to the multiple layers throughout your system. These tests will then immediately alert you when modifications impact application behavior–providing a safety net that reduces the risk of change and enables rapid and agile responses to business demands.

Policy Enforcement

Parasoft provides a complete Policy Enforcement solution, enforcing policies with executable rules that can be applied to WSDLs, Schemas, SOAP messages and any other XML artifact or SOA meta-data component. Once an organization has defined their policies to guide their SOA deployments, our solution recognizes these policies and enforces them throughout the development and QA process. For example, we verify schema and semantic validity for W3C and OASIS standards compliance, validate Basic Profile 1.1 for WS-I Interoperability compliance, and implement rules to enforce various other endorsed WS* Standards. In addition, we enforce compliance to best practices such as customized company guidelines, security, and maintainability and reusability.

SOA/WS

The Parasoft solution is "SOA Aware." This means that the rich metadata structure which drives the service transaction is managed and verified. As a result, both technical and non-technical users gain the ability to build, manage, and analyze quality tasks fast and effectively. In order to control how Web services are defined, developed, and deployed, companies utilize internal standards and policies, XML schemas, and various Web services specifications such as WS-I Basic Profile and WS-Security. Parasoft enforces design and development policies on SOA XML artifacts. This ensures consistent security, reusability, maintainability, and standards compliance.

Protocol

SOA messages must conform to both the transport protocol specification (such as HTTP 1.1) and the message protocol (such as SOAP 1.1). Parasoft helps teams exchange messages in heterogeneous SOA environment by automating testing of multiple protocols– including HTTP, SOAP/REST, PoX, WCF, JMS, TIBCO, MQ, EJB, JDBC, RMI, and so on.

Emulation

Evolving services in a distributed SOA environment, and across multiple teams, is a complex endeavor due to the interdependencies between the system and business processes. Parasoft automates server emulation across multiple environments quickly and easily:

  • You can interact with a real service by visually designing the traffic flow into a runnable scenario. Then, based on this interaction, automatically turn the resulting scenario into a stub that emulates the service behavior.
  • Service stubs are made available as a service so that different teams or business partners can collaborate on evolving their components within a SOA system.

Our intelligent stubbing for HTTP, JMS, and MQ services fills the necessary gaps across multiple environments quickly and easily so that end-to-end business scenario tests can be executed.

Continuous regression testing involves building and regularly running regression test suites to determine if code modifications impact application behavior. Continuous regression testing is a key part of Agile methodologies such as Extreme Programming; when practiced as part of continuous integration, it is known as "continuous testing."

Parasoft solutions enable continuous regression testing by facilitating the development of a robust regression test suite, as well as fully automating regression test suite execution and assigning each test failure to the responsible developer. A baseline behavioral regression test suite can be generated automatically to capture the project code's current functionality (e.g., with characterization tests). Once this baseline regression test suite is in place, you can incrementally improve its intelligence and value by:

  • Extending the automatically-generated JUnit, CppUnit, NUnit, or Cactus test cases.
  • Integrating your legacy JUnit, CppUnit, NUnit, Cactus, or HttpUnit test cases (or writing new ones).
  • (If the application can be exercised from the UI or messaging layer) Exercising the use case scenarios for the functionality you want to "freeze." Parasoft Tracer will observe the code behavior and capture it in unit test cases that can be run from a single machine, without accessing the production or staging system.

Parasoft can help you re-create stateful transactions to build complicated scenarios for complete functional testing. Additionally, test flow can be controlled by the defining logic in the test suite. This task is best suited for QA Engineers who work with clients on a daily basis and have the basic knowledge of each operation that is needed to complete a complex scenario

Policy Enforcement

Parasoft enforces established quality criteria and policies, such as security, interoperability, and maintainability, on the various business application artifacts within an SOA or Web application. Adherence to such policies is critical to achieving consistency as well as ensuring reuse and interoperability. As a result, you evolve a visible and sustainable quality process that delivers predictable outcomes. Our solution provides an integrated system for automated monitoring of code compliance across heterogeneous environments, core industry standards (accessibility, Section 508, etc.), and organization-specific policies (security, branding, etc.).

Web2

AJAX and Rich Internet Applications deliver a richer user experience as well as bring a greater level of complexity to the architecture and development of Web applications. Parasoft delivers an automated framework to isolate and test the unique components of an AJAX/RIA application and then aggregates those components into a comprehensive test suite. We validate the format and content of client/server messaging to ensure that the data is expected and correct. In addition, server messages are "frozen" and inserted in place of dynamic messages to validate that errors seen in the user interface elements are due to defects in the AJAX engine and not due to changes in dynamic application data.

Scenario

Parasoft's scenario testing emulates business logic or transactions that may occur during normal usage of the web service in order to demonstrate that the service works in context to the defined technical requirement. Leveraging the test cases from development, QA extends the scenarios to verify business requirements and use cases, testing positive, negative, and unexpected inputs.

Security

Parasoft supports testing Web services with security layers–at the transport level as well as the wire level. Additionally, we automatically generate tests to perform security penetration testing. By testing the Web service with penetration attacks and analyzing the responses, security vulnerabilities can be discovered and fixed earlier in the software development cycle. The following penetration tests are currently supported: SQL injections, XPath injections, parameter fuzzing, XML bombs, and external entities.

Parasoft's continuous quality practices promote the reuse of test assets as building blocks to streamline the validation of changing business requirements. This enables your team to execute a more complete audit of your business application, reducing the risk of business downtime and ensuring business continuity. Parasoft's business process testing capabilities also include the ability to test BPEL implementations, automatically creating test cases from vendor-specific BPEL deployment artifacts and allowing you to arrange these test cases into suites that reflect different aspects of testing a BPEL process. Organizations using BPEL to optimize and emulate complex business processes can now more reliably and consistently ensure the quality and completeness of these processes.

Parasoft solution's performance testing not only monitors the server's response rate with the specified number and mixture of simultaneous requests, but also verifies whether the test loads cause functionality problems. We facilitate performance testing by allowing you to load test your existing end-to-end functional test suites (which may span from the web interface, through services, to the database) according to preset or customized load test scenarios.

Preset scenarios can be used to verify robustness, scalability and durability. You can easily customize these scenarios to use different test cases, load levels, load distributions, and so on. You can also distribute virtual users across remote server machines to simulate extreme loads and/or test from different locations.

Support is also provided for load testing non-Parasoft components such as JUnits or lightweight socket-based components. This provides teams an integrated solution for their various load testing needs.

Performance

To help you collect network information and system performance data during load testing, Parasoft creates Windows Perfmon, SNMP, and JMX monitors. We also provide a detailed report option in which load test graphs, histograms, and errors can be saved as a detailed report that can be viewed after the load test completes.

SLA

Parasoft defines and enforces Quality of Service (QoS) metrics that are important for setting and measuring SLAs (Service Level Agreements) during development and QA. You can define success metrics before testing and set performance thresholds.

Continuous

Parasoft's Load Test Continuum plug-in allows you to access load test data through the browser interface. This enables you to track performance of services throughout the SDLC, thus discovering and resolving issues before they become a problem.

With Parasoft solutions, functional audit first consists of functional tests, also known as black box tests. Functional tests are designed as early as application use cases are created in order to verify application functional requirements at the highest level of abstraction. System functional testing begins upon the successful completion of integration testing-after all of the application modules and components are connected.

Following functional testing is load testing, which goes hand in hand with performance testing. The first objective is to verify that your system can continue to function correctly under both nominal and extreme loads (load/stress). The second objective is to verify that the response time under those loads is satisfactory (performance).

Parasoft solutions focus on creating extreme conditions for load testing for functional audit-loads that are twice as large as those under nominal conditions. Load testing is either performed on the same staging area as integration testing or, in some cases, conducted on the live application.

In the end, a functional audit validates that the expected end-to-end business scenarios are working effectively. When error prevention and continuous regression practices are performed consistently, the team can trust the quality of each component. This relieves QA from the burden of low-level bug-finding, freeing resources for more complete functional and load testing of the business application. As components of a business application become more complex and distributed, this building block approach is "must have."

SDLC quality metrics are fragmented across key systems such as requirements, build, and source control management. Parasoft aggregates and correlates this system data, delivering a comprehensive view of your development processes. This facilitates continuous process improvement, increasing productivity and reducing cost.

Visibility

Parasoft aggregates and correlates test data from the automated and manual tests performed at all layers of the application. By providing objective, real-time information about code quality and the testing process, this enables you to assess the effectiveness of your current testing efforts and determine what process improvements are needed to ensure security, reliability, and compliance.

Scanners

Since SDLC quality metrics are fragmented across key systems such as requirements, build, and source control management, Parasoft decided to simplify integration into core development infrastructure components. Our open, standards-based API allows your team to aggregate key quality metrics to gain control of the end-to-end quality process.

Workflow

Delivering quality as a continuous process not only requires technology to detect errors, but also technology to productively remediate the defects found. To help you ensure that your most critical issues are addressed promptly, Parasoft integrates error review/remediation practices into the team's existing workflow, as well as provides customizable severity prioritization, error assignment, and automated remediation techniques.

Analytics

Parasoft aggregates and correlates key quality metrics to deliver a comprehensive view of your development processes. In order to promote a continuous quality process, the Parasoft solution allows unique process views into your SDLC. This enables continuous process improvement, helping you boost productivity as well as software quality.

Parasoft's measurement "sensors" and reporting capabilities provide improved visibility into any testing strategy that your team has decided to adopt. This helps you determine if the strategy is delivering the desired result, as well as continuously improve it to ensure that it adds the maximum value to your development efforts.

Change Based Testing

Parasoft's change-based testing helps you optimize your testing efforts by identifying and executing only the test cases directly related to your most recent source code modifications. Not having to test the entire system after each modification yields tremendous productivity improvements.

Requirements Based Testing

Parasoft's requirements-based testing aligns testing efforts with business needs. In addition to graphically reporting requirement status as indicated by developers, we trace requirements back to test cases. At a glance, the team can gain an objective assessment of which requirements are actually working as expected, and which still require testing. This real-time visibility into true requirement status helps you prevent late-cycle surprises that threaten to derail schedules and budgets.

Dev Method Support

Organizations turn to development methods such as Test-Driven Development (TDD), Extreme Programming (XP), Agile development, Scrum, and so on to improve quality while enabling more rapid responses to constantly-changing business demands. Parasoft's measurement "sensors" and reporting capabilities provide visibility into the team's application of these and other development methods. This visibility helps you determine if the strategy is really being followed and whether it is delivering the desired results. It also helps you continuously improve the method to ensure that it adds the maximum value to your development efforts.

Parasoft solutions integrate with and collect data from other elements of a technology infrastructure, such as its source control system, automated build system, problem tracking system, and other systems, as needed to empower organizations with process visibility and control.

Based on the collected data, project metrics and status are calculated, and then reports are generated for review to provide process visibility and control. Team members can review and evaluate metrics, such as the number of tests passed and whether the style and coding rules are being followed. Architects can decide whether the groups are working on the correct requirements needed for the project and whether the groups are properly refactoring the code.

Because Parasoft solutions provide continuous feedback, implementation defects and other potential problems can be identified and corrected early-before they propagate through the product. The result of this on-going process visibility and control is improvement of an organization's ability to deliver quality software and to continually increase productivity.

Copyright © 1996-2010 Parasoft   T: 888-305-0041   E: info@parasoft.com       [Site Map]