The Agile methodology, with its breaking down functionality to user stories, has proven effective for independent applications, which are developed and maintained by a small team of experts. However, adoption of Agile for large scale Enterprise software development remains a challenge. The main challenge in enterprise software development is maintaining the large-scale integration. In this article, we will review ways for addressing the integration challenge. We will first describe what the integration challenge is, and then we will share how to overcome this challenge, using methods implemented by Amdocs Testing on enterprise communication software testing projects.

Integration Challenge Example

Enterprise solutions span across multiple applications. These applications could run on different platforms, use different technologies, and are often developed by different Vendors.

A Simple Story About Complex Integration
Let’s look at a real life scenario from the telecommunication world – a service provider, which we will call here ZCom, plans to launch a music service. The product owner has defined a user story as below –

As an existing customer of WorldCom, I want to add a free trial of music service so
that I can listen to music at no cost during Trial Period

Business analysts added the following exceptions to the normal flow:

1. If the user does not have a data plan, the user receives a message with recommendation of a data plan to be added.
2. If the user has already used the free trial in the last twelve months, he/she is given a message and directed to the paid subscription.

This is one of the several user stories that are associated with the launch of this service. The solution architect leverages existing applications to fulfill the story, as shown on figure 1.

Figure 1 – Example of a solution architecture for a simple single user story

As you can see, in the simplified version of the flow diagram for this user story, several applications operate behind the scenes to complete the provisioning flow that is initiated from a self-service portal. In a multi-Application, multi-Vendor scenario, which is very typical for large enterprises, each of these applications is managed by an independent scrum teams. In the example above, we have a scrum team for self-service portal, another scrum team for Billing System and yet another one for digital commerce. It is possible that some applications (e.g. the music vendor) may follow a waterfall approach for the pieces in their scope.

Integration testing stage

Typically, enterprise agile projects are in fact not pure agile, and end in production, following a waterfall like phase for integration testing, also called “release end testing”. This testing is done in an integrated test environment for all certified user stories. An Integration test team, independent of the application scrum team, tests the end to end integrated flows. That is, the integration testing contradicts the agile goal of ‘early and continuous delivery of valuable software that satisfies the customer’s needs’. The integration testing becomes a bottle neck and the objective is shortening it as much as possible. Yet, many defects are found at this stage.

In a project on a large communication provider in US, where Amdocs test team was engaged to do the integrated testing, the retrospective sessions revealed two reasons for defects hyping on integration testing:

1. The Application level user stories do not consider all end to end test scenarios.
2. Assumptions made by application teams at the interfaces are not synchronized.

In the example on figure 1, the Billing System and the Digital Commerce applications work under the hood. These applications do not directly interact with the user, although they play a major role in the background. The Billing system has to respond to the request from front end for eligibility. While it checks availability of a data plan internally for previously used free trial, it defers the decision to Digital commerce system which keeps records of free trials, opt outs etc.

The associated user story for Digital Commerce could be–

As the manager of digital commerce business for music service I would like the ability to define free trial period and track subscriptions so that a customer’s eligibility can be determined.

The user story above does not directly focus on the end user, the customer of the service provider. So while the functionality developed met the needs of the business owner of digital commerce, it missed issues related to flow of transaction through the system.

Beating the integration challenge

Amdocs Testing team mitigates these challenges by expanding the role of the centralized integration test team beyond the integration testing phase, in Acceptance Test Driven Development and Design (ATDD) methodology. In ATDD, all teams collaboratively discuss acceptance criteria, with examples, and then distill them into a set of concrete acceptance tests before development begins.

In the ATDD project the integration testing team includes test architects. Test architects start their tasks from the first stage of solution definition. The test architect, in close collaboration with Solution architect, defines concrete end to end integrated acceptance test scenarios for the feature. This is especially important for keeping backend applications relevant to user stories. End to end acceptance test scenarios are used to derive application level user stories and acceptance criteria in collaboration with the product owner and individual scrum teams.

On the agile development, the test architect works closely with the testers in scrum team to ensure that the story level testing keeps the integrated feature in focus. The test architect coordinates with testers in the scrum teams to ensure that end to end integration stories are indeed tested. End to end stories leverage interface testing with simulated or mocked up data to test realistic situations. The test architect works with testers in individual scrum teams to coordinate early drops into the integrated environment and to start testing at feature level (see figure 2).

Figure 2 – Centralized test team in ATDD agile project

Summary and Results

Agile for large scale enterprise software development imposes a challenge. The integration testing is a final bottle-neck phase, where many defects are aggregated to it. Amdocs implements ATDD methodology to ensure value to customer, early defect identification and smooth roll out to production. In projects adopting ATDD methodology 50% less defects were found in integration testing, compared to other similar projects with agile design and development.

Amdocs Testing unique methodology proved very effective. Testers on the team describe some of the result benefits:

1. Well understood end to end flow, which clearly laid out the cross-team dependency;
2. Test data and common test environments accommodating multiple agile teams contributing to the release;
3. Ability to flag, in advance, potential risks (functionality, timeline, cross team impact on common code base);
4. Well-orchestrated implementation of end-to-end flow, covering system test and UAT in one go, based on interactive build process, by keeping the process agile as well. This also helps overcome the challenge of accommodating the dedicated sprint for clean end-to-end run.
5. Ability to control defect fix implementation in a common environment shared across multiple agile teams. Based on the nature of the fix, it may be requested that the fix be tested in a smaller environment before promoting the fix to common test environment.

About the Author

Nandita Ramesh Nandita Ramesh is Director of Software Quality Assurance in Amdocs. She is responsible for Quality Assurance of Customer Care and Billing, Data warehouse, Business Intelligence, Predictive and Analytical Platforms based on big Data for a major Telecommunication company. She has 20 years of experience in software development, QA and training. She has been leading the transformation of testing from Waterfall to Agile in Amdocs.