4 Stages of Software System Testing

During system test, it's usually very hard to gauge progress. Most teams are still using the percent complete (how much have you tested) to report progress. While that's easy, that usually does not tell you correctly where you are.

Today I am starting to use the 4 stage system to tell the team where we are in System Testing.

The basic idea is that as you test, you will start by finding a lot of bugs, and as developers keep fixing these top level bugs, you will find more as you get deeper into the system. The bug list will grow on a daily basis. This is stage 1.

Then you getting to a point where you are finding less bugs, but the bug list is still big because the developers are working on high-level bugs and usually takes longer to fix. The bug list is slowly shrinking. This is stage 2.

As developers catch up, you finally get to a point when the bug list is empty and the developers starts to fix at a faster pace than the testers can find bugs. This is stage 3. The onset of stage 3 is when the rate of bug fixing finally go beyond the rate of bug finding. In real-life is not aways easy to spot to onset of stage 3 but if the bug list stays empty for a few days (assuming daily test cycle), the the project is in stage 3.

And finally, the testers will have a hard time finding bugs. And the bug fix rate is starting to drop as there are not enough bugs to be fix. Then, of course, at the end, the testers cannot find any more bugs for bug found and bug fixed are both zero at the end. Stage 4 starts when the rate of bug fixing slows down.