SOA Testing Challenges

Testing is a complex process for SOA applications. Don't wait until the last minute to plan a test strategy and develop or acquire the proper test tools, says , Chief Technologist at leading US IT consulting firm Perficient.

The most challenging characteristic of SOA Quality Assurance (QA) is the scope of executing an end-to-end system test. SOA is being widely used for application integration such as creating a composite application by service wrapping legacy systems. Composite applications are the most difficult to test.

Building an integrated composite application, such as integrating ERP and CRM, includes multiple systems as data sources and interfaces. The systems to be integrated will be outside the scope of control of the SOA project team. As such, test instances of the systems to be integrated may not be available to the team when needed. When systems aren't available for testing, the QA life cycle is disrupted. Another complication that arises involves the physical infrastructure requirements for testing the distributed environment. Test environments must be carefully planned or the interdependencies between systems will interfere with testing the software build cycles. (One build may be in integration test and another in system test.)

A traditional QA life cycle, involving unit, integration, systems and acceptance tests, simply won't work for SOA projects. SOA requires an agile methodology with incremental component development, builds and deployments. Enterprise-wide SOA transformation should be broken down into business integration domains to reduce complexity and risks. This incremental approach requires a matching incremental QA strategy, which places an additional strain on the environmental requirements and dependencies. Testing environments are required for long periods to test each iteration. Also, the frequency of regression tests increases to ensure that new builds (that reuse or enhance existing services) don't break the ones further along in the QA process or in production.

Framework for SOA QA Success

Integrating testing into the project life cycle starting at requirements analysis ensures that the test cases represent the actual user requirements rather than a design specification that may be defective. Test cases should be part of the requirements that drive system design; users, analysts and developers should understand them. Test cases may often be derived from use cases and event flows. Once requirements-driven test case creation (and maintenance) becomes part of the SOA project life cycle deliverables, then test case execution remains problematic. To this end, it's necessary for an SOA framework and test tool suite to provide the following functions to automate component and end-to-end system tests:

  • Test-aware interfaces: standardised interface and messages for testing
  • Message-based test automation: record, replay and management of test scripts
  • Simulation: the ability to simulate applications as part of a regression test
  • Validation: pass/fail verification at the component (messaging) and application (data store) levels.

A Service-Oriented Architecture (SOA) is a well-defined, if overused, concept. Given the SOA premise, one can consider the requirements of a message-based QA framework and toolset for SOA that can be built and designed as a set of services. Properly designed, the challenges of message-based SOA testing become an advantage. Automation of test execution can be realized through programmatic message interactions between test tools and the components being tested through a standardized test interface and message set. These messages can then be recorded for validation and replayed for regression testing and simulation. Once these services are available to architects, they can define the message interactions necessary to verify proper test coverage and execution (i.e., QA pass/fail criteria).

Green Hat’s GH Tester is a great fit to meet these requirements, and you can download it for a free evaluation. Existing GH Tester users can download the latest upgrade as part of their service.

Testing is a complex process for SOA applications. Don't wait until the last minute to plan a test strategy and develop or acquire the proper test tools.