Perhaps the single best known tester from the Indian subcontinent of Asia, Pradeep Soundararajan earned his reputation without the backing of a big company, without the impressive degree from an impressive college, and without writing a bestselling testing book (yet).
No, Pradeep did it the hard way. He had ideas worth listening to and repeating.
After a few years of testing for Motorola and McAffee, Pradeep became an independent consultant aligned with Satisfice in 2007. During his stint as an independent consultant he performed test coaching, wrote about testing, consulted, performed test management and mentored. His consulting work grew to the point that it could no longer remain a one man show, so in 2010 Pradeep co-founded Moolya Software Testing.
In his consulting work Pradeep practices exploratory testing, session based test management and rapid software testing. His consulting stories, experience reports, test reports and rants on his blog Tester Tested has made it a widely read and influential blog in software testing. (Matt Adds: on a personal note: Pradeep’s offbeat sense of humor can be extremely helpful in ‘jiggling’ people into ‘thinking differently.’ For those that agreed in the first place, it’s just pure entertainment.)
Matt found it especially interesting to be an American interviewing an Indian, so he let the first question come from a Russian living in the United States!
Pradeep Soundararajan will now answer your questions.
Question: What is your most memorable bug, testing experience or lesson learned? Elena Houser, District of Columbia (Formerly Moscow, Russia)
Pradeep: There was an interesting testing experience recently. I was on a consulting assignment away from home and the customer thought they had hired the best man in India to test their products – me. It’s a good ego boost when the customer tells you they hired you because they thought you are among the best in the country. I thought my goal set by the customer was clear: find as many important problems as possible for a product that we plan to launch. If it can pass through your tests, we’d be convinced of pushing it out. I tried helping them understand that passing through me is a heuristic, or rule of thumb.
About 9 days into the project, I was reporting bugs that I was finding and then all of a sudden I am called for a meeting where people say, “Pradeep, you are only finding low priority issues although they are high severity.” I was hit hard just like any other tester would have felt for that moment but then I said to myself, “Reaction to such a situation is what differs you from other testers. So be of value to the customer and don’t try to show that they don’t know to recognize your testing.” I accepted what they had to say about the issues I was finding and then with a soft voice replied, “I understand you want me to find higher priority issues but I kindly request you to understand that I am not as good a person to know what your priority is because you know the history of this product where I am new to it. Let’s strike a deal. You tell me what kind of issues you would consider as high priority and I can find them – if they exist.”
I went looking for the kind of issues they thought would be high priority and found so many of them that they were very excited and happy that they hired me. This not only gave them confidence in me but they also have been giving me repeat business.
Had I reacted any different way at that instant, or allowed my ego to get in the way, I would have ruined things. Thanks for asking this, I am going to detail it out in a blog post sometime.
“It’s a good ego boost when the customer tells you they hired you because they thought you are among the best in the country.”
Question: I’m curious about your experience with going off the script in testing. Say, for example, your organization is subcontracted to run specific test scripts. The bosses don’t want to run off script because they won’t be paid for the work. Have you faced this kind of a situation? Do you have any advice on where that thinking comes from and how to deal with it (to transition the thinking from script execution to be about adding value and mitigating risks?) Peter Walen, Grand Rapid, Michigan
Pradeep: I have faced the situation of my manager warning me when I was going off script. In 2005, I was even fired for going off script. This was a hard core six sigma process company that wanted a 98% test case pass rate to make a release. Their product looked great if we executed the scripts and yet the product did not perform well when we did a few tests outside of it.
The only way they could justify a release was by firing me, not that I was acting like a quality gatekeeper but I wasn’t helping them achieve the 98% pass rate. Until Jan 2009, they thought they were doing well and then there came news that the whole division would be laid off because the product wasn’t selling anymore. Hard way to learn that their customers weren’t executing the scripts of the 98% pass category. This is a Fortune 100 company. The story, if I looked back today, is all about how people were fooling themselves for five long years.
I think the value of executing test scripts depends on the context such as in providing compliance reports or fighting a lawsuit.
Regarding where such thinking comes from, it is obvious to me, that it is a lack of good education and practice in testing. I wish such people did get a copy of Jerry Weinberg’s Perfect Software & Other Illusions About Testing. I see the situation changing though. There are a lot of people who have been hands on testers, getting old or wise enough to be considered for senior management positions. They do recognize the value of not being scripted and going exploratory to focus on risks.
When people involved in signing contracts have done testing, they understand what is valuable and what is not. They are in a better position to negotiate for more value rather than just more test cases. If people don’t want to learn then there appears to be a hiring problem. Some people are going to remain adamant and not learn.
Question: Say you are working with a tester who is writing little tools. Should you work in the test group or the development group or should you transition at some point? To follow up on that, at what point do testing tools need to follow company standards for code style, format, review, version control, and so on? Jason Koelewyn, Philadelphia, Pennsylvania
Pradeep: If a tester is writing little tools, should the tester move to a dev group or test group? To answer that, I’d like to know what the tester does after writing those little tools. I think such a tester should be in both dev and test groups because the tools being written would help both teams. If I were the tester’s manager, as long as the person is doing a great job, it wouldn’t matter to me which group the tester would like to be associated with. Whatever is comfortable for a person who is performing well should determine the right group for that person.
I am struggling to answer what are the appropriate standards to follow because, in my own experience, I have seen the same things yielding different benefits to different teams. For instance, in the context of starting something fresh, every experienced person would want to incorporate coding standards but if someone made a mess and the person is hired to do a cleanup and maintain the future releases, they might have to adopt a different strategy.
Question: What’s the first thing you would say to a brand new tester? Catherine Powell, Boston, Massachusetts
Pradeep: My initial comment to a new tester would be that everybody who has been testing for awhile, has at some point, gotten lost. You too will get lost…. (a long pause) but if you are constantly learning, practicing testing, networking with good testers and focusing on your skills, you will help yourself get back on track.
Question: As a tester interviewing (whether in an actual interview, or just starting out in) a new software development organization, what are some things to look for and questions to ask that can give you confidence that the team is committed to building a test culture including implementing automated testing if appropriate? Sean Stollberg, TriCities, Washington
Pradeep: Testing culture of a team is very important and I am glad you asked a question on those lines.
I would start by asking, how do the employees learn? What kind of freedom do you have at work? How much do you think your work is valued by your colleagues and management? How do you handle the situation when your team member is not working to your expectations? What kind of books have you read? What blogs do you read often? Can you give me examples of the team taking responsibility for a failure? What are your goals? What does a good day in the office look like and what does a bad day in the office look like? How often do the good days occur and how often do the bad days occur? How many people have quit from this team? Do you go to lunch together (not the company sponsored team lunches)? How often is that?
You may expect people to give answers that are not true and if you discern that is the case, you can still get to know the culture. All these questions are heuristics anyway. Having made some bad decisions early in my career, I have come to a point where I can “smell the culture” when I walk in for an interview. I can’t explain why some places have a pungent smell and others are aromatic. A culture that is purposely scented to appear ”good” is more dangerous.
Alternatively, considering the kind of questions the interviewers are asking may help in making some kind of inferences into what they study, what they might value and so on. I would ask an interviewee if they have heard of heuristics & oracles. They can sense what kind of testing I value by my follow up questions.
Follow up Question: Conversely, what are some things to look for that make a software development organization a dangerous or bad bet for a tester? Sean Stollberg, TriCities, Washington
Pradeep: I have been asking myself a question whenever I visit a software company: Are these people testing just because they should or because they see a value in it? Going in as a consultant, I give myself opportunities to learn the answer to that question as quickly as I can.
Also, when testers hired in organizations are told to wait for instructions from the developers to perform their tasks, such organizations are a BIGNO for good testers. The worst scenario is when companies have testers solely because they need someone to be responsible for poor quality. In such places, testers are first made to feel as though they are the God of Quality without letting them know that they are being taken for a ride.
Question: How do you handle the increased pressure to reduce costs while improving testing? (Do more with less.) Also along those lines, how do you suggest responding to the release it now/minimal viable product mentality that is commonly known as the “facebook effect”? Anonymous
Pradeep: Is there a pressure? If there is, who is it directed at? I know it might surprise you that I ask if pressure exists. I have seen how people play around with it. They compare their testing to much worse testing and show that they are doing well. The generation today is excellent at making their work look great. The pressure is more on customers these days than the vendors to avoid falling into such traps.
Question: I’ve enjoyed your recent postings on testing culture in India. Can you identify something as the next big thing coming from that community? If yes, can you tell us about it? Lisa Crispin, Denver Colorado
Pradeep: Thanks for enjoying those posts. The next big thing is difficult to answer but let me share some interesting things that are happening. In Bangalore testers are meeting on a monthly basis to share ideas and get better. We had testers from Chennai traveling 350 kilometers to attend these meets because they didn’t find such events happening in their city. In places where I was invited to speak across India, I kept challenging the people saying, “We (Bangaloreans) are ahead of you. Are you going to catch up? We don’t care about you because you should care about yourself” (making it look provocative).
I happened to challenge a gathering of testers in Noida part of the National Capital Region comprised of Noida, Delhi & Gurgaon. The testers meet up was scheduled and there were about 100+ testers who turned out. It was a great meet up from what I gathered from the tweets. If this continues to happen, we are going to see more testers talking to each other and identifying what is good testing rather than just following best practices. This can potentially cause a revolution in the way a lot of testers in India test.
Of course, you know about weekend testing. What started as just a group of 5 testers from Bangalore has expanded to similar meetings across 4 continents.
I also hope the next big thing becomes the testing services company I have co-founded with Santhosh Tuppad, Moolya Software Testing Private Limited (www.moolya.com) where we offer the kind of services good customers are looking for.
Question: Often the comments of return on investment are quoted as being why testing gets a lower rung on the ladder compared to software development or even marketing. How can we help move the conversation forward that investing in testing will result in a solid return on that investment, both in the short and long term? Michael Larsen, San Francisco, California
Pradeep: I hate the term ROI, specifically because many people understand ROI in a different way than I do. Every tool vendor I encounter sells their tool by using ROI as the key. The ROI’s they use are those associated with money savings and these are the ones who compare humans and tools. They say, “If these tests were to be executed manually it would have taken much more time and therefore cost more; our tool does it in a jiffy”. Many business people of the previous (err, current) generations were envisioning the cost savings associated with tools and were buying them big time. In just a couple of years, people will find it hard to sell it that way because I see many testers educating themselves about value and moving up the ladder.
So, instead of helping them, we need to help ourselves by giving ourselves more time whilst constantly educating ourselves to better communicate what we do and how it helps business.
Question: If you could be the ‘King of Software Testing’ for a year, what changes would you make? Anonymous
Pradeep: I am already the king of software testing and working hard to be the emperor. (Note to self: A lot of people might be reading this, let me try pretending to be humble.)
Jokes aside, I don’t think the King of Software Testing can exist. Even if he does exist I would choose to be a student of software testing forever rather than be the King of Software Testing.
By being a student of software testing, I still can make changes to the way people test. I just hope I continue to change myself to make myself more useful to the testing community.
Question: Do you think it’s possible for sapient Indian testers and their consultancies to work the outsource model and test software for countries outside India? Anne Marie Charett, Ireland, United Kingdom
Pradeep: Well if non sapient testing companies have been doing good business in India, sapient testing and “brainual” testing companies should also be able to do it. That’s a bad way to equate 🙂
I understand why you might be asking this question. Most people who outsource their testing work to India have been doing it for scripted work but only recently they have realized that there appears to be some kind of a change in asking for value. The traditional test case – pass versus fail – hasn’t helped customers much.
At least in Moolya Software Testing Private Limited, we ask, “Is there a problem here?” and use the Heuristics & Oracles approach to test software. For us what matters is that stakeholders’ questions about the product are answered. Thankfully, we practice being of service to the project and not as an obstacle to it. That way, we help our customers move faster.
I have seen testing services companies trying to hold back information as much as possible so as to keep pushing the project for a longer duration. What they don’t realize is that by fooling the customer they are fooling themselves. I am equally surprised at how some customers don’t recognize that.
About the Author
Matt Heusser A consulting software tester and software process naturalist, Matt has spent the past 12 years or so developing, testing, and leading in dev/testing of computer software. Beyond the work itself, Matt has had notable roles as a part-time instructor in Information Systems at Calvin College, a contributing editor to Software Test & Performance Magazine, the lead organizer for the 2008 Workshop On Technical Debt, and most recently as Senior Editor for the “How To Reduce The Cost Of Software Testing” book project.