“Hi Mike, can you please let me know the minimum amount of time you’re going to need to test the system? Given the level of dev, I thought two weeks would be enough, is that fair? I need to cut some time from the schedule and testing looks like the obvious place.”
In my ignorance, this was part of an email I sent to the test lead of a system I once was the project manager on. As you can probably guess – the response I received was less than impressive and suitably uncomplimentary.
What happened next was a lesson that has stayed with me for a (very) long time.
I went to a meeting with the test lead and the test manager to “discuss” the problem and ended up having one of the most productive meetings of my career.
Instead of being hostile, Mike and the test manager, let’s call him John, used the time to educate me on the fundamentals of testing and explained why testing is so unpredictable. I was so impressed by what I learned I’ve recreated it here.
In response to my query about cutting the testing time, Mike explained that the testing schedule isn’t able to be determined until they get the software from the development team. He added the estimate of time required they gave me in the beginning was just that – an estimate. How long the testing will actually take depends on the complexity of the system, as well as the level of difficulty the testing team expect to encounter when they receive it. It might be longer than the estimate or it could be shorter. It all depends – doesn’t that sound familiar?
As an example Mike told me about a critical bug in a previous piece of software that took four weeks to be fully reproducible and it turned out that the software was clashing with a popular social media tool. This pushed the release of the software out by two months but the company was grateful to have the bug fixed pre-production rather than being inundated with complaints. Something like this isn’t predicable but the schedule needs to allow for something like it happening.
I did ask about providing additional resources to the test team as a way to increase the speed of testing but Mike explained that while non-testing testers are great, they wouldn’t be able to help in this instance.
John then talked about how the test team’s role was to ensure that the software that came out of the company was of excellent quality and that we as an organisation had a reputation of producing better quality software than our competitors. He said that while he wasn’t willing to jeopardise that by releasing incompletely tested software there were options.
- Release partially tested software
- Release a stripped down version of the software that would be upgraded at the next release, or
- Release the product “late”.
It was up to me and my boss to decide which one of those options we were prepared to go with.
I left the meeting with a much greater understanding of the testing process and felt able to justify to my boss why we couldn’t cut the testing time. It also meant that the next time I was involved in a technical project I could share what I had learned, ensuring the test team didn’t get hassled by another ignorant project manager.
Oh, and for the record, a complete version of the software was released on time as per the original schedule thanks to heroic efforts from Mike, John and the team!