Many of the folks we feature on ask the tester are consultants with big names and public faces, and that’s good. Still, we try to achieve a balance; we also want to hear from practitioners.
If we’re going to put on a practitioner, how do we know they are good?
Well, how about someone from a company so flawless in its IT execution that you never hear their name in the news?
Here’s one: Lowe’s companies. You know, the big buildings that sell lumber, hardware, and lots of other things.
You could probably also guess that they have a corporate office, HR department, have to pay a lot of salaries, have a complex supply chain, keep a website running, and have an IT department with more than a few testers.
Meet Mike Lyles, QA Program Manager for Lowe’s Companies Inc. Like many folks in QA, Mike has been at one company for over a decade (nearly two), quietly doing the work while slowly gaining promotion from help desk to system analyst, project manager, QA Manager, and now QA Program Manager.
For this month’s theme on test management, we thought he was the ideal fit.
Michael Lyles will now answer your questions.
Question As a test manager, how do we get our testers to actually work on improving their personal test approaches and knowledge within whatever test context the organization has? – Jon Hagar, Mountain tester; Hot Sulphur Springs, CO, USA
Mike Lyles This is always a struggle. What I have found in my experience is that it’s even more difficult to get a testing organization to follow the same methodologies and practices. We have guidelines, templates, processes, and standards, yet teams get caught up in the consistent drive of the projects and lose focus on following those standards. With any occupation, the key is to keep yourself in continuous improvement. There is a quote “HR is not looking out for your career growth – it’s up to you.” This is so true, especially in testing organizations. I encourage my team to continually read up on new processes and methodologies, as well as look for opportunities to build upon their skillsets by studying testing materials (books, online repositories such as www.softwaretestpro.com magazines, articles, etc). When you cease studies, you cease to improve. And finally, I encourage my team to think outside the box and to come prepared to suggest new testing approaches – we don’t guarantee that all suggestions will get accepted, but we will surely discuss and determine if it’s an improvement opportunity. Certifications are an awesome way for team members to stay in sync. Our organization went through training courses in early 2011, and each of our team members became ISTQB certified as a result. This investment by our company was instrumental to ensuring that everyone ‘spoke the same language’
Question What is unique about testing at your org compared with others in the industry? How is testing changing? Is it changing? Are your concerns as a manager different than they were as a test practitioner? If so, how are they different? – Lanette Creamer, Seattle, Washington, USA
Mike Lyles Our company started our testing group in late 2008. Before then, testing was conducted by two developers – the one that developed the code, and another developer that would conduct a ‘second test’ on the changes. We have grown so much in three short years yet we are still evolving as a world class testing organization. I am not sure I would characterize our organization as unique compared to others in the industry. In fact, I feel we are still learning – trying to establish ourselves among the other groups in IT. However, I do feel we are making tremendous strides currently, with plans to improve our processes in 2012 in a way that will make us very effective.
As far as changing, the world changes so fast, you either follow it and stay ahead or you become irrelevant. Stephen Covey once said “Nothing fails like success.” Things you do today that are above par and surpassing others in the industry will be below average and irrelevant in a very short time. Therefore, it’s our responsibility to keep relevant and ensure we move with the best practices that are being presented by so many of the testing experts in the field. I am fortunate to be involved in major initiatives right now, as a Test Environments and SCM manager, to do just that for my organization. Our first focus is to ensure our testing environments are stood up, set up accurately, and ready for the first day of testing. In concert with this initiative is our focus on Test Data Management – ensuring that not only our data is accurate and precise, but that it is in sync among all the various test environments we work on today. Environments and Test Data are so critical to the success of testing, and if an organization is not focusing on these to streamline the processes and move to steady state in these two areas, they will always have issues at test execution time. Lastly, we are making a significant step this year in Software Configuration Management. While this is not typically a testing organization role, it is critical to ensure that the code promoted from development to QA and eventually to production is 100% accurate. Incorrect code, data, or environments are three elements that can make or break a testing organization’s execution.
To your last question – I have held many roles in the organization. I was a developer & tester for a long time. I was fortunate to be part of many testing efforts even before we had the testing group. And I feel that the major difference in concerns of a test manager verses a test practitioner is a focus on commitments. In a well run team, the test practitioners should feel confident focusing on the preparation of test cases, preparing for and conducting test execution. A test manager should be focused on orchestrating all the external factors that could keep the test practitioner from achieving those results – such as coordinating with the Test Environment, Test Data Management, and SCM teams to ensure that everything is available and ready for the test practitioners to do their job. Additionally, while the test manager should empower the team to monitor and govern the entry & exit criteria to move between development testing to QA testing to UAT and production, the accountability for auditing and governing these practices must belong to the test manager. Any role where a person is a manager should be taken seriously. Your team is looking to you for answers and guidance, and it’s critical to be responsive and supportive at all times.
Question What would you advise a tester to do now if they wanted to become a test manager in the future? How did you become a test manager? – Lanette Creamer, Seattle, Washington, USA
Mike Lyles This is a great question and one I get asked by my senior test engineers often. I strongly feel that each step we take in our careers can prepare us for the next phase. There is so much to be learned from being in the trenches as a test engineer. You get the opportunity to have hands on, “front row,” visibility to the things that work and the things that do not work in the testing lifecycle.
The thing I challenge my team to always focus on is to become evangelists for the testing practices that our organization has established. And I suggest that they submit them to memory so vividly that they don’t need to refer to a standards document or reference book to speak it. The team should be in sync and speaking the same language, and MOST IMPORTANTLY, they should collaborate to ensure that there are no contradictions in the practices by one team or test engineer verses another.
The last thing you want to hear in a major project or initiative is “these rules, standards, practices, entry / exit criteria are not the same as the last project I worked on.” If you can get the team to speak consistently, the positive perception and respect of the organization will grow exponentially in a very short time. Lastly, most teams expect their test engineers to be heads down, running the tests, reporting the results. I always suggest to team members wanting to move toward management that they begin building the relationships with the key stakeholders of the projects they are working on – because as they move into a management role, they will need the support from each of these teams to accomplish their goals fully. Therefore, I always suggest to my up and rising test managers that they focus on not only the methodologies, but their approach on how they will enforce them with minimal to no friction among the teams.
Question Do you experience “communication gaps” to other managers or execs/stakeholders – i.e. a simplified perception of what testing can and can’t do (is and isn’t)? If so, how do you work with that perception? – Simon Morley, Stockholm, Sweden
Mike Lyles We all know that it is difficult to enforce standards and practices. We are the ‘Highway Patrol’ of IT. We set the rules for testing, we post them, and if the teams are not following them, we step in and work with the teams to correct the mistakes. The goal of the test manager is to ensure you do this with tact, respect, and a focus on collaboration – and that we maintain respect among the organization in the process. None of us like to see a Highway Patrol, in our rearview mirror, pulling us over for a violation – but we most always respect them for the job they are doing to make the roads a safe place to travel. The same applies to the role of the test manager (and ultimately the testing organization). You have to be prepared to show the value of the testing organization and the benefit of the methodologies being enforced.
Question And a follow-up, if I may: How do you discuss or present your “testing story/message/report” to stakeholders (or other non-testers)? Do you (or your stakeholders) separate this from ‘feelings’ about the product being tested? – Simon Morley, Stockholm, Sweden
Mike Lyles I like your statement of “perception of what testing can and can’t do.” This is critical to be clear and concise early in the project for the expected roles of the testing organization, the development teams, and any other external teams to the testing group. And it is important that the key stakeholders are aware and agree with the set roles and responsibilities. The way our organization has ensured this collaboration is to schedule and conduct a kickoff meeting early in the project and to walk through what our expectations are for entry and exit criteria and what conditions would cause us to not be able to move from one phase of the project to the next.
Once testing has started, the focus should be on the defects and how to resolve them – the key is to ensure this is not “we vs. they” – we drive for a collaborative team approach to determine the reason for the defects, the resolution, the timing of that resolution, and the plan for resolving those issues effectively. The earlier an organization can detect and recognize a defect, the lower the costs involved in the resolution. Tracking defect leakage and where the defect was detected (i.e. in Unit Testing, Component, System Integration, or UAT) will be critical to helping the team to look for ways to identify defects earlier in the process.
Question As a QA Program Manager, how do you think you can best impact a tester’s role? (Speaking of which, how do you differentiate between test program management and test management?) – Janet Gregory, Calgary, Canada
Mike Lyles In our organization it has proven critical, as a test manager, to provide immediate assistance and support to the testers to ensure they have everything easily accessible to conduct their jobs. Our focus for this year has been to provide fully functional test environments, concise and accurate test data that is mapped to the requirements for the project, and to react quickly to any escalations or needs that the team has along the way. Another thing we focus on is to ensure that the testers do not have to debate the methodology or standards for the testing organization. They should be able to focus on their role in creating, executing, and reporting on the testing efforts and the discussions between the testing organization and the other teams should be made at the management level.
To your question on test manager vs test program manager – in my situation, my title didn’t change, only my role, as a QA Manager and QA Program Manager. However, my responsibilities grew. Instead of being a direct Test Manager for a specific project or projects, I was in position to lead Test Managers who were working with major program initiatives for the company. We had many testers across these major strategic programs (over a hundred). And in this situation, these programs were all interrelated. Therefore, I spent a lot of my time bridging the gap between the programs, recognizing the interdependencies, monitoring the risk – especially if issues or delays in program A impacted program B, C, or D.
Question Your management asks you to come up with metrics that will show the value your team brings to the organization. What metrics do you use? – Yvette Francino, Colorado Springs, Colorado
Mike Lyles This is one of my favorite subjects when it comes to testing. “What gets measured gets done” is so true when it comes to testing organizations. I had the privilege of meeting Michael Bolton (the testing guru not the singer) earlier this year at a conference and he was kind enough to sit with me over dinner and share his thoughts on metrics.
Michael made a statement that has stuck with me throughout – he said “First we must agree that what we are talking about is a Measurement Program. Metrics is the raw data that feeds a Measurement Program. You wouldn’t characterize a book as ‘words.’ ” I agreed and since that time I have referred to our reporting as a Measurement Program.
Building a world class Measurement Program is a monumental task. Anyone who says it is “just data” has not been involved closely enough to realize the importance of measurement. And you have to be prepared to present your measurement program in various ways to different audiences. For instance, there are things a test manager, development manager, PMO, or business stakeholder will want to see that an IT VP or Senior VP will want to see at a more summarized level.
Regardless, I will share with you some of the things at each level that I feel are important for any measurement program:
- Project Level: Need to show the defect leakage, time to resolve defects, and the overall metrics on the effectiveness of the testing at all phases of the project. Also, if your entry and exit criteria require that certain levels of defects (i.e. critical defects) cannot exist to move to another phase of the project, it is critical to be able to report the status of the critical and high defects and the timing for when they will be resolved.
- Management Level: For this reporting, you should focus on a testing dashboard which will display the status of the testing efforts, any impacts to timelines, and any mitigation to the risks of not meeting those timelines.
- Executive Level: This is where your reporting cannot have the raw data. Executives will want to see the health of the project, but also to recognize the areas where testing was effective and saved time and resource dollars along the way. Also, this is an opportunity for the team to showcase areas where major issues were diverted and to give praise to the teams that were responsible for assisting in the mitigation.
- Business Level: For this reporting, it is critical to first sit with your business stakeholders and discuss what is important to them in measuring the project success. Different business stakeholders may see value in various ways. But once you have this information, the reporting here should showcase the benefits of the testing, and a focus on how we saved time and money in the process.
At the end of the day – the goal of the measurement program is to provide every one of the recipients of the reports the information they need to fully understand the status of the project and to be aware of any risks. You will know that you have successfully implemented an effective Measurement Program when the selling of the benefits of the testing organization is coming from the stakeholders and not the testing organization. And it all hinges on the accuracy and presentation of a measurement program. Question As a manager, what challenges have you faced to keep close and accessible to your team(s)? What have you done to overcome these challenges?” – Abigail Buell, South Bend, Indiana
Mike Lyles I assume my challenges would be the same as any manager in similar environments. The more you are responsible for, the more meetings and emails you will be accountable to participate in. Being accessible is always a challenge but every minute spent with your key players is critical to the success of your team. I say “key players” because as a manager, your focus should be on building a team where you have your star players supporting you in areas that are the most critical to your deliverables and your company. Without this, you will spend all of your time in the details and the time you need for strategy, planning, and coaching the team will diminish. My response to this challenge is to first ensure those key players are in position to support the team with me. And the one thing you should always coach the team on is the information that is being shared with their peers and management. A VP once said to me during a status update “Mike, tell me what I NEED to know, not what you WANT me to know.” I have never forgotten this advice, and when you and your team follow this mantra, you will find that meetings are more efficient, status updates are more concise and clear, and unexpected surprises will cease to exist.
Question I’m a QA Manager, wondering where to climb next. What does a QA Program Manager do and how often do testing skills actually come in to play in that role? – Eric Jacobson, Atlana, Georgia
Mike Lyles This seems to be common for most all professionals. Reaching a management position means preparing for executive management positions to move upward. And as we know, the number of positions decrease as you make the climb. Opportunities are so open, however, as a QA Manager. In my company, we are still growing as a QA team, and I’m fortunate to be part of many opportunities to build out something new and innovative for the organization. My suggestion to you would be to not wait until you take a step above QA Manager. Begin today looking for opportunities to be innovative. Look for areas where the organization could see a high ROI if a new way of thinking or process change could be instituted. Volunteer for things you are passionate about, and give every ounce of hard work and dedication to making it successful. Words are cheap…deeds are dear. Let your work speak for you, and you will be noticed for this in the end. Did I mention you should look for INNOVATIVE opportunities?
But I will stress again, if you don’t have the appropriate skillsets and leadership in the Test Manager positions, you will find yourself drowning in the details and unable to function as a Program Manager. Team member selection is critical!
Question As a leader, I find a lot of value and joy in encouraging team members to bring their individuality to the table. I think it’s very important to rely on the unique strengths that each person contributes, rather than forcing people to “do as I say.” How do you tap into the more personal side of the members of your team when managing such a large group of people? Is it possible? – Michele McCubbins, Buchanan, Michigan
Mike Lyles It has been said that it’s much easier to pull a string than push it. And I can’t agree with you more on the value of coaching and mentoring your team. People join companies for the company and opportunity, and they most always leave one because of their boss and how they feel they are valued.
You have to strike the balance between folks who are capable and willing to take on strategic roles verses those who want to be given clear direction on the tactical deliverables. You need both of these personalities to survive today.
Nothing satisfies me more within the team than a team member approaching me to say they’d like to offer a suggestion for a better way to do something. And I was fortunate to work with one employee who brought me problems, but always brought me multiple solutions to them. I felt like I was handed a menu, and all I had to do was make the best selection.
Also, if you gain nothing else from my responses, the one advice I can give that I hope you take to heart is that being a manager does not mean you are always the mentor or coach. I have learned that you can learn a lot from your team, and I have had situations where I felt I was going to be the mentor, and to my surprise, I was the one that was mentored.
I like your note about tapping into the personal side of the team. We spend at least one third of our days working with our teams, and it’s critical to ensure they are happy, sure of your expectations for them, working together collaboratively, and delivering to the expectations. I learned early on that being someone your team can depend on is very important. And making them feel important and valuable to the team is your only hope for survival. It has been said “people don’t care how much you know until they know how much you care.” If your team feels valued, respected, and important, they will move mountains for you. I take pride in making sure I talk to my team members about their feelings regarding the efforts we are responsible for, and even if it’s a very short time due to the size of the team, I make an effort to spend time with each of them as much as possible. The challenge is finding the balance, and how you can work this in to your daily rush of emails, meetings, and emergencies. However, the more time you find to spend with your team and appreciate them for their hard work, the higher the quality and timeliness of their deliverables.
Question What aisle are the hammers in? – David Hoppe, Grand Rapids, Michigan
Mike Lyles At least one question I know I will be 100% right on – that would be aisle 64 in Tool World. And my painful attention to detail forced me to call 5 stores in our company and ask them this very question to validate it was true. I’ll be honest, I expected it to be different in various stores, but the resounding response from our helpful Lowe’s store employees was “aisle 64.”
Your question first made me laugh – thanks, I needed that – but then it made me think of something motivational I could say to close out this set of questions. Abraham Maslow said “It is tempting, if the only tool you have is a hammer, to treat everything as if it were a nail.” How many times have we, as testing organizations settled with a tool that we know is not the best choice? How many times have we known the best solution but decided we had what we needed to get the job done.
My challenge to each of you is to think differently. Look for ways to improve your organization. Technology changes so fast today – and when you see something that is successful for another organization, learn from this and look for opportunities to do the same with yours. Inform your management team of new ideas and ways of thinking, and keep yourself fresh with the latest tools, methodologies, and best practices. You don’t have to start the next Apple, Google, or Facebook. You can change the world we live in one day at a time by reaching beyond your limits.
My very special thanks to Matt and STQA for allowing
me to be part of this article. And I wish each of you the
best in your future!
Connect to me on linkedin at http://www.linkedin.com/in/mikewlyles. Or on Twitter: @mikelyles
About the Author
Mike Lyles Mike Lyles is a Quality Engineering Program Manager with over 22+ years in IT: development, PMO, and Software Testing. His experience spans functional testing, test environments, software configuration management, test data management, performance testing, test automation, service virtualization, building testing organizations, defining processes and methodologies, and standing up measurement programs
Mike is an international/keynote speaker at multiple conferences, and is regularly published in testing publications and trade magazines. Mike’s passion to help others improve and grow in the field of testing, leadership, and management is his key motivation. You can learn more about Mike at http://www.linkedin.com/in/mikewlyles, www.MikeWLyles.com or http://about.me/mikelyles
Mike’s comments, blogs, articles, and opinions are his own and not those of his employer.