How the QA team DOES own Quality

I’ve spent the last 4 blog posts discussing how everyone except the QA team owns quality so, to cap this series off, lets discuss what role the QA team has in the delivery of a quality product… because it’s an important one.

The QA Team

As I stated in the opening of this series you are here to accurately measure the status of customer quality regularly in a repeatable fashion throughout the development cycle. The words here are critically important.

**Accurately –** You are responsible for designing the series of tests that the product must pass in order to be released. You won’t be able to test every configuration and stress test possible before the product is release so you have to make sure you are testing what’s important. Just ask the person that had to test the iPhone 4 antenna… and left off actually holding the phone like most users. I’m sure they had thousands of hours of testing that went into that release… they just missed what was important.
Measure – There are lots of ways to measure quality but the most important are going to include:
- Test Plan pass fail reports. When tests are executed do the pass or fail. If they fail why do they fail?
- Known Bug counts by severity. All products ship with bugs so it’s not just important to know how many there are… but at what severity the known issues are throughout the cycle so that the team can fix what’s most important first. The point is to make sure people are aware of the bugs.
- Code Complexity – If you are shipping software you need to be able to communicate where it’s complex and look to reduce complexity BEFORE it’s developed in the first place by reviewing development plans, tickets, or user stories. You should push back on requirements that will make the product untestable and unnecessarily complex.
- Test Counts VS tests remaining – Everything needs to have coverage… this really measures your work backlog.
Regularly – It’s meaningless if you stand up at the end of development and say 50% of our tests fail. Its your job to put quality in everyone else’s face so they can make informed decisions about adding more feature work, the time required to lock down the release, and work that must be done. The more frequently you can run tests the faster you will find issues and the less it will cost to fix them.
Repeatable – The test plan and test cases must be structured so that it can be run the same way consistently. This means the more automated and structured it is the better. You should be able to hand a test plan and automation over to any new person on the team and they should be able to repeat your results. Even the automated tests should be manually checked, but having a nightly build report from all the unit & integration tests is critical to finding issues fast.
Customer – You should be reading every customer reported issue. Those are one that snuck out. It needs to be fixed and have a test case added to it. Customers are doing you a favor by telling you about something and its your job to stand up and make sure those issues are addressed. As a bonus – nothing beats a few good rounds of ad-hoc testing by creative people trying to break things.

Ok, this concludes this 5 part series on product quality. It’s obviously something I’m pretty passionate about and I hope you enjoyed it. If you are digging back for the first 4 posts in this series to see how other roles own quality.. here they are: