Spread Knowledge

CS605 - Software Engineering II - Lecture Handout 15

User Rating:  / 0

Related Content: CS605 - VU Lectures, Handouts, PPT Slides, Assignments, Quizzes, Papers & Books of Software Engineering II

Statistical Control Techniques – control charts

Same process metrics vary from project to project. We have to determine whether the trend is statistically valid or not. We also need to determine what changes are meaningful. A graphical technique known as control charts is used to determine this.

This technique was initially developed for manufacturing processes in the 1920’s by Walter Shewart and is still very applicable even in disciples like software engineering. Control charts are of two types: moving range control chart and individual control chart. This technique enables individuals interested in software process improvement to determine whether the dispersion (variability) and “location” (moving average) of process metrics are stable (i.e. the process exhibits only natural or controlled changes) or unstable (i.e. the process exhibits out-of-control changes and metrics cannot be used to predict performance).

Let us now demonstrate the use of these control charts with the help of an example.

Let us assume that the data shown in the following table regarding the average change implementation time was collected over the last 15 months for 20 small projects in the same general software domain 20 projects. To improve the effectiveness of reviews, the software organization provided training and mentoring to all project team members beginning with project 11.

Statistical Control Techniques – control charts

In order to determine whether our change in process had any impact, we use control charts.

This data is first presented in the graph form as follows:

control charts

We now develop the Moving Range Control Chart as follows:

  1. Calculate the moving ranges: the absolute values of the successive differences between each pair of data point. Plot these moving ranges on your chart. (the dark blue line)
  2. Calculate the mean of the moving ranges. Plot this on the chart. (the red line)
  3. Multiply the mean with 3.268. Plot this as the Upper Control Line (UCL). This line is 3 standard deviations above the line. (the light blue line)
  4. To determine whether the process metrics description is stable, a simple question is asked: are the moving ranges values inside the UCL? If the answer is yes then the process is stable otherwise it is unstable.

This chart is shown in the following diagram:

Moving Range Control Chart

This chart is then used to develop the individual control chart as follows:

  1. Plot individual metric values
  2. Compute the average value for the metrics values - Am
  3. Multiply the mean of moving average by 2.66 and add average computed in step 2 above. The result is Upper Natural Process Limit (UNPL)
  4. Multiply the mean of moving average by 2.66 and subtract average computed in step 2 above. The result is Lower Natural Process Limit (LNPL)
  5. Plot UNPL and LNPL. If LNPL is less than zero than it need not be plotted unless the metric being evaluated takes on values that are less than 0.
  6. Compute a standard deviation as (UNPL – Am)/3.
  7. Plot lines 1 and 2 standard deviations above and below Am.
  8. Applying these steps we derive an individual control chart.

This chart may be used to evaluate whether the changes represented by metrics indicate a process that is in control or out of control. For this, the following 4 criteria zone rules are used.

  1. A single metrics value lies outside UNPL
  2. Two out of three successive values lay more than 2 standards deviations away from Am.
  3. Four out of five successive values lay more than one standard deviation away
  4. Eight consecutive values lie on one side of Am.

If any of these tests passes, the process is out of control otherwise the process is within control.

Since none of the test passes for the data in our example, our process is in control and this data can be used for inference.

We now analyze our results. It can be seen that the variability decreased after project 10. By computing the mean value of the first 10 and last 10 projects, it can be shown that the remedial measure taken was successful and resulted in 29% improvement in efficiency of the process. Hence the process changes incorporated were useful and bore fruit.