Friday, September 25, 2009

Bug - Curve based on used process - or - How to measure the influence on the quality by the used process.

Bug - Curve based on used process - or - How to measure the influence on the quality by the used process.
The question I try to answer here is how the used process influences the shape of the bug-curve of a project.
In a waterfall approach we have a clear differentiationtion between the different phases. First we decompose the system to be developed and specify everything, design it into details and then start with the implementation. After finishing the development and, if we are lucky, also the unit test, the developed components will be given to the test team to start the validation against the functional specification. Then, depending on how big our system is a second test phase will follow. The so called system test. This second test phase is responsible for the test of the whole system, meaning the integration of all SW-components with the HW and to test some none functional parts as e.g. documentation, ... If we have a more advanced development process the test team will already start to test some components although the whole development is not finished. As we start with the test the bugs found by the test team will start to inflow.As the developers are still busy with the development they will not be able to work away all the bugs coming in at such a fast pace. So they will start to pile up. The shape of the bug-curve will start to raise. Then after the first test phase the developers will start to catch up and the curve having passed a maximum will start to decend again. Then the big bang starts again as soon as all components are thrown together and the system starts. A second bump is going to appear in the curve putting the developers under stress as the system should be already ready and not so many bugs shall be found again as the deadline has to be kept.
What we can see at this graph are the adjusted curves of two project; one done as a waterfall the other one as agile-development.
So how does such a curve look like when an Agile development process is applied? Having different basic principles (e.g. Always have a running system) given, the curve shall look differently. Let us try to validate this from a theoretical point of view. A bug means that there is something in the system that is not working as specified. What does this mean from a Agile process point of view? Well I would say that certain criteria that were given, where not followed or not in place (e.g. only features that fullfill the DONEness criteria are finished, automated test in place, ...) or it is something that we oversaw when we broke down the requirements from the customer or a new requirement. Beside this, another important aspect is to involve the test as early as possible and not to have this strict separation between development and test. So based on this two topics: 1. basically higher criteria for release of functionality into the test and 2. early test of the finished functionality; we will expect to have 1. less bugs and 2. earlier feedback to the developer about the quality of their development. That would mean the shape of the bug-curve shall look differently, more flatened.
The idea behind the thinking about this curves is that by looking at the shape of the curves from different developments it is possible to depict a certain pattern and with this drive conclusions out what kind of process is being used and how good it works. It seems that the shape of the curve is independent of SW, of system and even of company that is developing. So the curve could be an indication how good our process is containing the quality of our product, as the area below the curves gives an idea about how good/stable our actual product is at that actual moment.
So this was the theoretical approach. What I have found out is that although I did not expect such a good mapping of the theoretical thoughts with the practical implicationit is exactly what came out when running Agile development and comparing also here the Bug-curves from different systems and companies.
From this I would clearly draw the conclusion that from a bug curve we can derive the used process and that agile definitively helps us to maintain the bug curve flat and to deliver more stable and reliable systems. Having a lower bug-curve alows us also to deliver out our product, when ever needed.


  1. just linked this article on my facebook account. it’s a very interesting article for all.
    Agile Process

  2. Hi, interesting article.
    Was the two projects identical?
    How many developers had been working on those projects?
    Were there any significant difference between the resources, employees?