Same process metrics vary from project to project. We have to determine whether the trend is statistically valid or not. We also need to determine what changes are meaningful. A graphical technique known as control charts is used to determine this.
This technique was initially developed for manufacturing processes in the 1920’s by Walter Shewart and is still very applicable even in disciples like software engineering. Control charts are of two types: moving range control chart and individual control chart. This technique enables individuals interested in software process improvement to determine whether the dispersion (variability) and “location” (moving average) of process metrics are stable (i.e. the process exhibits only natural or controlled changes) or unstable (i.e. the process exhibits out-of-control changes and metrics cannot be used to predict performance).
Let us now demonstrate the use of these control charts with the help of an example.
Let us assume that the data shown in the following table regarding the average change implementation time was collected over the last 15 months for 20 small projects in the same general software domain 20 projects. To improve the effectiveness of reviews, the software organization provided training and mentoring to all project team members beginning with project 11.
In order to determine whether our change in process had any impact, we use control charts.
This data is first presented in the graph form as follows:
We now develop the Moving Range Control Chart as follows:
This chart is shown in the following diagram:
This chart is then used to develop the individual control chart as follows:
This chart may be used to evaluate whether the changes represented by metrics indicate a process that is in control or out of control. For this, the following 4 criteria zone rules are used.
If any of these tests passes, the process is out of control otherwise the process is within control.
Since none of the test passes for the data in our example, our process is in control and this data can be used for inference.
We now analyze our results. It can be seen that the variability decreased after project 10. By computing the mean value of the first 10 and last 10 projects, it can be shown that the remedial measure taken was successful and resulted in 29% improvement in efficiency of the process. Hence the process changes incorporated were useful and bore fruit.