Will Process Improvement Save the Day? Probably Not. You Need
If you’ve been asked to start or grow a testing team, you’re probably wondering, “Where do I start?” Since test teams today are expected to find more bugs in less time, in increasingly complex environments and with less guidance than ever before, the answer can seem elusive. The typical answer is “Why, process improvement, silly.” We would like to suggest a different path.
“Why, process improvement, silly” tends to lead to little arrows and diamonds and squares. Process improvement efforts tend to be led by a small of pool of “elites” who design processes for “resources” to execute. Eventually, the elites go away and some tester has to sit down, design the tests and execute them. At that point, the defined process is irrelevant compared to the skill and talent level of the person designing the tests. Process improvement doesn’t specifically improve skills, so growth is generally left up to the individual.
As leaders, we can’t magically improve the skills of our staff members, but we can lead them toward improvement. Skills improvement offers a distinct contrast to process improvement: It involves engaging the whole team for the purpose of leading people to do better work. Personally, we find the skills-based approach refreshing, redeeming—and all too often ignored.
Now that we have an answer to “Where do I start?” we need to address how to do skills improvement. The “how” here involves basic leadership: mentoring, inspiring, coaching—things we do every day. Formalizing things a bit, picking which skills you want to work on and investing time and effort in leadership will have a direct impact on your team’s ability to deliver.
We would like to help with that skills selection process by providing a list of key skills to think about—and some guidance as to when to emphasize each of them. Instead of a complete, exhaustive list, we’ll work with a manageable list of things you might want to think about. We will organize those things into categories representing each stage of the management catchphrase of “forming, storming, norming, performing.” (See sidebar, below.) With only a handful of skills listed per growth phase, you could tackle them one at a time, or one per person.
Along the way, we’ll also offer recommendations as to where to go to learn the skills, including books, classes and Web sites. We’ve tried to make that list realistic—training your company might actually send you to, books your company might actually pay for (or books you can get at the library and read for pleasure!) and Web sites that offer free, high-value information written in plain English.
This article offers advice on what roles and skills should be introduced or nurtured at each stage. We also offer recommendations as to pertinent literature along the way.
Forming the Team
If you are trying to fill headcount and figure out budgets, then you’re probably in the “forming” stage. Unless you can finance a dream team, forming the team will be all about tradeoffs. More than likely, your choice of testing team members will come from a pool of fresh college graduates or hungry, unemployed software developers, or by promoting or hiring nontechnical people into a nontechnical black-box testing role.
If that leaves out seasoned test professionals, you certainly should consider bringing in a consultant who can provide ongoing guidance. If you can’t afford or can’t find test professionals, and you can’t afford a consultant to help, then skills development and training opportunities become even more critical.
Before discussing critical skills in the forming stage, first we must debunk a myth: that testing should exist solely as a promotion path into development. The logic is seductive: Testers can’t mess much up, because they don’t write production code. And when testers are finally promoted to development, they will know the system they will code on, and they will have a healthy respect for errors.
We understand that hiring a tester with some development skills may round out the team, but we strongly advise against staffing the team entirely with people who view testing as a transition job. If testing is a transition job, this makes development “the real skill to learn.” Testing is a craft in and of itself, with a different skill set, and it must be treated that way.
For each stage of the model, we would like to talk about a few key skills to nurture and encourage at that stage. At the forming stage, it’s foundation skills you’re looking for in team selection. Skills to think about during this stage include the following:
Critical Thinking. Critical thinking involves conceptualizing, applying, analyzing, synthesizing and evaluating information to reach an answer or conclusion. This is nothing less than the ability to learn a system and attack it to see if it holds up; we emphasize critical thinking for testers at all times.
You can test anything—from software under evaluation to requirements. Doing that testing requires skills in conceptualization, analysis, evaluation and such—critical thinking skills that are taught in the liberal arts, philosophy, law and hard sciences. Our advice: Recruit testers with these skills, or create an aggressive training program to develop them.
The traditional way to learn critical thinking is to read Aristotle, and most community colleges have a night course in the philosophy of symbolic logic or rational thought. There are plenty of Web resources on logical fallacies, but for light reading, try Isaac Asimov’s “I, Robot,” Robert Heinlein’s “Orphans of the Sky” or Robert Asprin’s “Phule’s Company” (Ace Books; reissue ed., 1990).
Risk Management. Consultant and author Lee Copeland claims that testing is applied risk management, in that we have a limited amount of time to test an infinite combination of things. Testers need to determine and communicate risks to management, and they must prioritize the risks, deciding how much to invest in testing what.
Few people make the connection between risk management and testing. We suggest recruiting staffers with at least basic risk management skills, or adding these capabilities to your team early on; it can make your life much better. “Waltzing With Bears: Manag-ing Risk on Software Projects,” by Tom DeMarco and Timothy Lister (Dorset House, 2003), is the principal book on software risk management. Another book that offers practical hands-on guidance is “Risk Management in Software Development Projects,” by John McManus (Butterworth-Heinemann, 2003).
Basic Test Techniques. Presumably, most of your staff members understand and can apply boundary values testing, decision tables and exploratory testing. But if every candidate you hire thinks these terms are gobbledygook, consider special training or a contractor.
If “training” consists of “go read a book,” we can recommend one: Lee Copeland’s “A Practitioner’s Guide to Software Test Design” (Artech House, 2003); he also offers tutorials and workshops at www.sqe.com. The Software Test & Performance Conference (stpcon.com) also covers test techniques in considerable detail.
There are some excellent test training Web sites: satisfice.com, testing .com and developsense.com all have good, free information on test techniques. Testingeducation.org is a federally funded site with entire books, lectures and courses available online for free. Finally, Paul C. Jorgensen’s text “Software Testing: A Craftsman’s Approach” (CRC Press, 1995) is worth investigating. His coverage includes presenting the choices available to you in test design.
Domain Knowledge. The team needs to have someone who has a deep or personal knowledge of the specific field or domain the software addresses. For example, developers of a golf course ERP system (don’t laugh, they exist) might look to hire customer service representatives, its trainers or possibly ERP system users. If your test team doesn’t have domain knowledge, you can develop it through exposure and example.
Storming
During the “storming” stage, different ideas compete for consideration. Team members vie for position as they attempt to establish themselves in relation to other team members and the leader, who might receive challenges from team members. The conflict involves questions like: Does the testing instill quality? Do the testers provide information to decision-makers? Do they make go/no-go decisions on product releases? At this critical point, our recommendations for skills development include people skills, such as negotiation and facilitation.
This storming occurs for several reasons. First, the larger organization may believe that many of its quality issues will go away because of the newly formed testing team. Second, the customers of the testing team (the internal developers and the organization) will have a new step on the path to delivery. The organization may want to skip the testing phase when pressured by a deadline, or perhaps the developers will stop performing unit tests because testing is now someone else’s problem. Overcoming these little tests of definition requires an agreement on some hard statements like:
- Software testing is not debugging.
- Software testing is not automatically a threat to a developer’s career.
- The new, additional bug reports are an indication of improving software.
- Software testing is a planned, disciplined process.
- Software testers will not solve all your problems overnight.
Until there is some consensus on what testing is and how it’s done, the conflict can tug at productivity and hurt the team politically.
Bad political moves can sink a team before it’s really started; Ben Franklin is credited with the statement “We must all hang together, or assuredly we shall all hang separately” at the signing of the Declaration of Independence. For our purposes today, hanging together requires having a guiding testing philosophy that will provide a bond among testing team members, developers and customers. The time to build that philosophy is during the storming stage.
How can a team and its organization develop a testing philosophy?
In our experience, this happens when team members have some sense of ownership and control. That’s right: We are suggesting that the team itself define what it can do and how it will do it. That sense of ownership and self-control will kick-start self-management and self-learning, making skills im¬prove¬ment a habit—a way of life.
To succeed here, consensus-building is needed to resolve conflicts and enable group decision-making. This can be done with the following building blocks.
Systems Thinking. Where critical thinking is destructive in nature, systems thinking is essentially constructive. A skilled critical thinker can identify an intellectual shantytown—say, for example, a bogus maturity model. It would then take systems thinking to provide a better, alternative model in its place. A good place to start is “An Introduction to General Systems Thinking,” by Gerald Weinberg (Dorset House, 2001), though the first volume in Weinberg’s Quality Software Management series, “Systems Thinking” (Dorset House, 1992), is also worth reading. Robert Heinlein’s book “Star-ship Troopers” and Orson Scott Card’s books, such as “Treason” and any title with the word “Ender” in it, also contain applied systems thinking.
Formal Facilitation Training. Facilitation is the art of bringing out the best ideas of a group and building a consensus around those ideas. While this is certainly useful for officially anointed leaders, we recommend it for each member of the testing team; they often become leaders in certain areas because of their specific roles.
While facilitators should remain neutral in their discussions, organizations will seldom budget for a full-time facilitator in addition to regular staffing. A member of the testing team can facilitate a discussion of testing practices despite a lack of neutrality. We have found that a trained facilitator understands the value of maintaining a level playing field in building consensus, as well as the techniques involved.
There are many books and training programs on the subject of facilitation, but we will limit our recommendations to three: The works of Esther Derby and Naomi Karten, and the AYE Conference (www.ayeconference.com).
Negotiation Training. While testers are trying to ship the best possible product, project management is trying to hit deadlines, and senior management just wants to make money. Most people view negotiation as “your desires or my desires,” with a winner and a loser or a compromise that pleases no one.
Modern negotiation practices provide a better way: helping different groups find mutually beneficial ground. For instance, it’s natural that the testing team and the developers will disagree on processes for changes, particularly for “emergency” fixes. The discussion should not start with a focus on the mechanics of the process, but the intent of the process. In essence, testers need to follow Covey’s advice in seeking to understand before being understood. This is easier said than done, and it requires practice. To learn these skills, we recommend Fisher and Ury’s classic “Getting to Yes: Negotiating Agreement Without Giving In” (Penguin; 2nd ed., 1991).
Agile Techniques. Many of the conflicts that arise during storming involve what your organization “should” do during this phase, and the resulting expectations. Does “disciplined” mean “methodical and process-driven”? Does “robust” mean “document-centric”? Do you desire quick releases? Feature or requirements freeze? The agile movement (www.agilemanifesto.org) offers a different approach, moving away from heavy process and toward quick delivery of “good enough” product, with a focus on human relationships. Some agile techniques apply to the testing world, and Lisa Crispin’s book “Testing Extreme Programming” (Tip House, 2003) covers that ground.
Additional Testing Techniques. In the storming phase, when the team is struggling to chart its own destiny, sending team members to conferences and training might be ideal. Does the team want to perform load testing? Are they a testing team, or are they responsible for some deeper meaning of “quality”? Be aware: If your team doesn’t try to define this, someone else will.
BZ Media sponsors the aforementioned Software Test & Performance Conference, and Software Quality Engineering sponsors numerous tester training events (www.sqe.com). If you can’t find room in your budget for formal training, “Lessons Learned in Software Testing,” by James Bach, Bret Pettichord and Cem Kaner (Wiley, 2002), is a collection of wisdom from three renowned experts.
Norming
Finally, the team has defined itself. Armed with a clear vision, we now have to execute. This means the individual staffers are gaining expertise, learning what works and what doesn’t work by doing it. Because things are finally happening, senior management is probably starting to crank up the heat for better results, shortened test cycles, etc. To deal with these new challenges, focus on the following skills during the “norming” stage:
Test Automation. We believe that before you can automate anything, you must first define it, perform it for some time and truly understand the issues involved. For that reason, we’ve waited to list test automation until after storming is over. Once your team is established, system and acceptance test automation can further enable quick releases and incremental development. To help realize effective test automation, we recommend Dan Mosley and Bruce Posey’s book, “Just Enough Software Test Automation” (Prentice Hall, 2002).
Another phrase for the process of “piecing together” test automation is “homebrew” test automation; Bret Pettichord (www.pettichord.com) has done some impressive work in this area. On the developer testing side, Micheal Schwern offers a free talk (www.wgz.org/chromatic/perl /Test-Tutorial.pdf; the audio can be accessed at www.perl.org/yapc/2002 /audio/schwern-test-tutorial.wav .mp3). Java developers may want to read something by Kent Beck.
Software Development. Test automation often requires programming skills, so you may pursue these in parallel. Encouraging your testers to “get technical” will also help them communicate with the development team.
For what language to learn, you could try to pick up a language the development team is already using (consider a night class at a community college, or just buy the book and ask for help), or pursue a test automation language. Perl is a proven test automation language; it is powerful, easy to learn and popular. Randal Schwartz and Tom Christiansen authored an excellent book called “Learning Perl,” now in its third edition (O’Reilly, 2001).
Effort Estimation. A great deal of testing is simply doing the best we can within the time we’ve got. Instead of testing until the clock runs out, actually predicting a test schedule is a surprisingly high-maturity thing to do. It requires an understanding of how good is “good enough,” how long it will take developers to fix bugs, the percentage of regression errors that typically shake out of fixes, etc. It’s possible to learn more and more about these things, to do better prediction—and, in fact, it’s a skill. Rex Black (rexblackconsulting.com) and James Bach (satisfice.com) have two very different approaches to test estimation. We cannot recommend one single approach; you’ll have to play with both and customize the approach to your environment.
Requirements Inspection Skills. If the requirements are vague, incomplete or inconsistent, the software developers are out of luck before they’ve started. Then again, testers can test more than just software. Getting your testing team involved in the requirements review process can prevent defects and shorten your time-to-market.
Exploratory Testing. Testing the software by using it and bending it in different directions is the essence of exploratory testing. This technique can find an entirely different set of bugs than traditional requirements-based testing approaches do. Bach has done some great work in this area, and he offers regular tutorials and training sessions.
Usability Testing. Beyond the charter of many traditional testing groups, usability testing affects the customer’s ability to use the software, which can have a huge impact—both on productivity for internal software and on return on investment for external software. Even though it is “off-charter,” usability testing might be a natural fit, one that senior management will understand and attach importance to.
For good reading on usability testing, try Joel Spolsky’s free book on user interface design, available at www.joelonsoftware.com. Andrew Starling also has an interesting article on “true” usability testing posted at wdvl.internet.com/Authoring/ Design/UsabilityTesting/.
Monkey Testing. “Monkey testing” involves a program that points, clicks or types random things at random places on the screen. Tying that up to record/playback and playing back the last 10 seconds before a crash can be surprisingly educational, for a relatively small price. Once you’re past the small initial programming investment, monkey testing is a fast and cheap way to explore the software and find defects that might otherwise remain hidden. Advanced monkey testing moves into model-based testing, where you describe the various states the software “should” travel through and then run tests to take random paths. Harry Robinson, perhaps the best-known evangelist of model-based testing, has done considerable writing and speaking in this area (www.geocities.com /harry_robinson_testing/robinson .pdf).
Other good skills for this stage include design, feedback analysis, performance testing and security testing.
Performing
The “performing” stage does not indicate perfection, only that the team is stable and is jelling. Perform¬ing is a stage of continued growth, and the team can now work toward a higher level of self-management. You’re finally getting to the point where John R. Katzenbach and Douglas K. Smith’s “The Wisdom of Teams: Creating the High Perform¬ance Organization” (Collins, 2003) can have direct application.
For example, each team member may become an expert in a specific area or two and can take responsibility to mentor others. In short, the performing stage is a period where team members should continue to follow Covey’s advice to “sharpen the saw,” or, as the U.S. Army advises in its Army Field Manual 22-100, “Military Leadership”: “You must constantly stress bonding, learning, training, teaching, coaching, caring, and teamwork because they lead to cohesion. Your goal is to develop the full potential of every soldier in your unit so that individually or collectively they could continue and complete the mission in your absence.”
That is the fundamental problem of organizational learning: It requires that we step away from the busy business of production to work on something more long-term. Stepping away means investment, which costs time and money now and pays off later—something that far too many managers and executives are unwilling to deal with.
Welcome to management. Nobody ever said this stuff was easy.
Some ideas on what to invest in during the performing stage: a continued emphasis on critical thinking; performance metrics; test architecture; assisting and pairing with developers on unit tests; and programming skills and model-based testing.
What to Do Tomorrow
We have tried here to not describe a laid-in-stone learning and growth model; different teams have different challenges and grow in different ways. Instead, our goal in this article is to provide a flexible framework, along with some things to think about as you analyze your team and list its goals. Sooner or later, you will have to sit down and set some of those goals, and make plans and changes to meet them.
As you do this, start small. Analyze where your team is along the continuum of “forming” to “performing.” Pick an appropriate, specific skill for the team, or at most one skill per person. Focus on skills with a high bang for the buck that will be visible to a high-level stakeholder.
Find a book, class, Web site or conference for the staff to learn from, and provide the opportunity for them to practice their newly acquired skills. This doesn’t have to be hard; many of the books we recommend above are an afternoon’s read of escapist fantasy.
Make sure the person who learns a skill is somehow rewarded—even if that’s nothing else than the pride in learning something new, and the implicit understanding that personal development is just something we professionals do around here.
Then do it again and again and again.
C.S. Lewis once said, “If in your working hours you make the work your end, you will presently find yourself all unawares inside the only circle in your profession that really matters. You will be one of the sound craftsmen, and other sound craftsmen will know it.”
Developing sound craftsmen, or skills improvement, is a fundamentally different approach to our responsibilities than process improvement. It means focusing on making our people better, instead of trying to idiot-proof the process.
The great thing about building an inner ring of skilled craftsmen is that this is essentially about building a community—something Tom DeMarco has described as the essence of true management, a noble calling.
Then again, it’s no big deal. Just something we professionals do around here.
H3. SIDEBAR: 4 STAGES OF TEAM BUILDING
• Forming: At this stage, the team is being assembled. While introductions are made, team members will not necessarily have a comfort level with each other at this point. However, their specific roles can be defined at this stage.
• Storming: In the “storming” phase, different ideas compete for consideration. As team members define their roles, conflict is inevitable. The storming does not necessarily occur at a personal level—it’s more due to different viewpoints finally being examined. It is also a byproduct of the learning process each team member is going through. This may be, perhaps counterintuitively, the stage at which there can be the most effective, long-lasting form of team building.
• Norming: Team members are now comfortable in their roles, and there now exist working relationships based on agreed-upon standards.
• Performing: At this stage, team members have become naturally synchronized in thought and action. Although they have been achieving prior to this, now they have exhibited that the sum has become greater than the parts.
It’s important to note that this is only a model—an imperfect approximation that enables a high-level discussion. Further, this is not a process maturity model such as CMM or TMM—it is merely a model of how relationships evolve on a team. We found this model helpful; more information is available online at www.chimaeraconsulting.com/ tuckman.htm.
About the Author
Patrick Bailey