It’s Like Conducting an Orchestra!

Imagine this:
You are a testing professional newly hired by a mid-to-large size company. Any industry. You are responsible for the testing of all of the company’s internal corporate systems—HRIS, Finance, ERP, CRM, CLM, Sales Management, and Knowledge Management to name a few. You have just been tasked with providing the testing strategy and test management for a large-scale commercial-off-the-shelf (COTS) implementation. At first you think, “Hmm. A COTS vendor application that has already been developed, tested, boxed up, and released?!? This job is going to be a piece of cake!”

Not so fast, buckaroo.

This is where I found myself earlier in my career. I took a role within the Corporate Systems – IT division of a private mid-size company, where I was responsible for the testing of all corporate applications, which were mostly COTS systems. The corporate systems department focused on implementing new systems, upgrading existing systems, and integrating multiple systems. Not only did I learn how to test the apps—mostly from trial and error, oops—but I also learned that testing COTS systems in a corporate systems integrated environment is quite a bit more complicated than I had originally thought.

Here are some of the lessons I learned as well as a high-level strategy that can be adopted to test almost any COTS system. The strategy I use is to divide the COTS test effort into separate “test tracks” where each test track is managed individually with its own team of test resources and its own project plan. The individual test tracks roll up into your master test effort, including the master test effort project plan. Yes, I said, “project plan.” It is my experience that in order to manage test budgets, test timelines, and test resources on a large-scale, complex COTS test effort, it’s important to view yourself not only as a tester, but as a Test Project Manager. Your role is really like conducting an orchestra. While each musician learns his part, it is the conductor’scresponsibility to bring all the musicians together and ensure they play cohesively to produce beautiful music.

I’ve organized this article into three parts: Part I is about how testing COTS systemsis differentcthan other test efforts; Part II focuses on the various test tracks I recommend for all COTS implementations; and Part III provides some keen insights into how to better manage the COTS >test effort.

PART I – The Instruments: A COTS System

Before preparing to conduct an orchestra, it is important to gain an understanding about the instruments involved. For our purposes, we’re going to discuss one instrument, a generic COTS system.

Lesson #1: With COTS systems, you are not testing the actual product itself as if you worked for the vendor company that developed it.

As we all know, software never arrives 100% defect-free. If you have worked for a software vendor in the testing division, then you also know that there is a balancing act between the quality of the system and time-to-market delivery. With vendor products, many times the time-to-market aspect outweighs the value of a truly vetted application.

Continue Reading by Downloading This White Paper Now

About the Author

Virginia Reynolds Virginia Reynolds is the founder and principal consultant for Insight Quality, a boutique IT consulting firm. She is an IT Implementation Specialist focusing on high-risk, high-profile, large-scale multi-system application development projects with a focus on strategy, process implementation, and management within the high-risk areas of IT services delivery such as test strategy & management, requirements strategy & management, change management, configuration management, and environment strategy management. She has spent her career creating test divisions, managing testing for software product companies, leading the overall test strategy for application development at Fortune 100 organizations, and initializing & implementing continual process improvement. Virginia’s guiding principle on every project is the use of “smart flexibility” where she resolves challenging issues with efficient solutions by bending the traditional “rules” without compromising quality.