Agile has become more common, but functional testing often remains a manual process because during Agile iterations/sprints, there is simply not time to automate it. In this article, I will introduce a method for testing and automation called Action Based Testing (ABT) and illustrate how you could use it to achieve effective and timely automation in Agile projects.
Action Based Testing
There are various sources where you can read more about Action Based Testing in general. Let me summarize the key principles here:
- not one, but three life cycles
- the emphasis on test design to drive automation
- the role of automation engineering
Life Cycles
It is common to have testing and automation activities positioned as part of a system development life cycle, regardless of whether that is a Waterfall or an Agile approach. ABT however regards three distinct life cycles. Even though they have dependencies on each other, in an ABT project they will be planned and managed as separate entities:
- System Development: follows any SDLC, traditional or Agile model.
- Test Development: includes test design, test execution, test result follow-up, and test maintenance.
- Automation: focuses solely on the action keywords, interpreting actions, matching user or non-user interfaces, researching technology challenges, etc.
Test Design
The most important property is the position of test design. It is seen as the single most enabling factor for automation success, much more than the actual automation technology. In ABT, it is considered crucial to create a good test development plan first, in which so-called test modules are defined. Each such test module should have a clear scope that is different from the other and is developed as a separate mini project.
A test module will consist of test objectives and test cases. The test objectives outline the scope of the test module into individual verbal statements defining what needs to be tested in the module. The test cases, which are only defined when the test module is created, are defined by a series of actions describing operations and checks. Each action consists of an action keyword (or simply “action word”), followed by arguments. The arguments can specify input values or expected output values.
Automation
In ABT the automation activity is not part of the test development. It focuses solely on implementing the actions. Once implemented, an action can be re-used numerous times in any of the test modules.
This separation of automation allows for quick and Agile creation and maintenance of automation. And it allows for an efficient use of the automation engineering role (often a different person), who will focus on creating necessary technology and interfacing with the system or systems under test, not having to worry about the subject matter under test.
To get success in Agile projects I would generally recommend to start technical automation preparations early. Identify and research any technical challenges, such as graphics, messages, API’s etc, and address them. Think about environments, including cloud, virtual machines, etc, that the sprint teams might want to have to progress. Also make sure to test automation, including any developed actions, before testers start using it.
Agile Test Development
When using ABT the question of how to timely automate testing divides into two different, separate, task: (1) test design leading to test modules and action lines, and (2) automation: making the actions work.
Using the example of a scrum project where system development is organized in sprints, testing activities fall into three timelines:
- Testing within the sprints
- Test development prior to sprint development
- Testing after development has finished
Test development planning, in which the test modules are defined and planned, should in my view also happen in an Agile project. I feel the best way is to do it in step with the product backlog planning. In fact I regard test modules also as products in their own right. Test modules, including a definition of their scope, would go into sprints together with the user stories.
Testing in regular sprints
The most common practice nowadays is, and will remain, to develop and execute tests as part of sprints. Just like user stories test modules related to the user stories can be allocated to sprints as well. Then in the sprint a good rule of thumb could be to specify the test modules at the same or similar level as the user story, and to do it in the early phase of the sprint. The test module can then also help the team to understand in more detail the subject matter of the user stories.
In many cases this will involve a relatively high level of actions, which then later in the sprint can be further implemented using action definitions or scripts. If the automation infrastructure, like necessary technology, is in place before the sprint, it should not be too complex to get the test modules to be ready when the functionality under test becomes available.
Unit tests written with xUnit functions are an invaluable asset, but in the ABT approach one would like to consider options to re-use and extend their reach across the lines of addressing single functions. By defining test modules for unit tests and assigning them to actions, they can be strung together more easily to test with a wider variety of values and include other parts of the system under test, either during a sprint or later on.
Test development prior to development sprints
In the ABT method the use of actions, in particular high business level actions, allow for the development of tests with a non-technical focus on business functionality, often simply called “high level tests.” Such tests stay away from details in the UI and focus on business transactions, like requesting a home loan, or renting a car.
Higher level tests can be developed early in a project. These tests don’t have to wait for a system development sprint since there will be limited time to carefully understand business functionalities and create appropriate tests for them.
The number of, and whether or not business level tests can be made, depends on individual situations. In general, I would recommend the following:
- Have as many business level tests as possible, as they add great value to overall depth and quality,
as well as being resilient against system changes that do not pertain to them. - Use the high-level test design step in ABT (where the test modules are identified) to determine what can be done early on in business level tests, and what needs to be completed in detail tests as part of development sprints.
Testing after sprints
Once sprints for individual system parts have finished and the parts come together, usually more testing will be welcome to ensure quality and compliance of the entire system. Also, tests may be needed to retest parts of systems that were not touched by system changes and to confirm that the new system parts integrate well with old ones. This testing can be part of hardening sprints or done separately.
In this stage it may also be advisable or necessary to repeat tests for numerous platforms and configurations, and in the long term tests may be re-used to test various versions and spin-offs of a system.
All this “after-testing” is where it pays off most to have well developed test modules and fully automated actions. I generally recommend thinking about and planning testing and test automation early in a project, including a mid-term and a long-term perspective. Make sure to understand what tests are needed and when, and what is the best approach to get them. Well-designed automated tests can have a high value by themselves, in addition to the systems they’re testing.
About the Author
Hans Buwalda An internationally recognized expert in testing, Hans Buwalda is the pioneer of keyword-driven test automation, an approach now widely used throughout the testing industry. Originally from The Netherlands, Hans is now CTO of California-based LogiGear, directing the development of the successful Action Based Testing™ methodology for keyword-driven test automation and its supporting TestArchitect™ toolset. Prior to joining LogiGear, he served as project director at CMG (now Logica) in the Netherlands. Hans speaks frequently at international conferences on concepts such as Soap Opera Testing, 3 Holy Grails of Test Development, Testing in the Cold, and Jungle Testing. Hans is coauthor of Integrated Test Design and Automation.