One mistake most people, including policymakers, make is to over-react or under-react to data points on a run chart. What they forget is that repeate
d measurements of any process performance will naturally yield different values. For example, even if the government did nothing to make roads safer, subsequent annual death rates would still indicate a better or worse performance. It means that interpretation of improvement data on run charts must follow statistical probability–based rules.
Second, avoid interpretations based on a single, or most recent data point. You need a minimum of 12 data points to analyse if changes introduced into a process lead to improvement (in our case study on traffic deaths, we’ve 20 data sets).
In addition, when dealing with run charts, terms such as ‘shift’ or ‘trend’ have statistical definitions based on level of significance needed to provide an objective statistical threshold (p<0.05) indicating whether changes are leading to improvement, or degradation.
As such, to identify with confidence whether changes lead to improvement, I look out for these four things: a shift, trend, run or astronomical points.
If you look at the graph below titled, “Traffic Deaths in Kenya from 2001 to 2020, you’ll notice that from 2004, when Michuki Rules were introduced, there are six consecutive data points all going up until 2009 when you the first drop.
A “trend” refers to when five (5) or more consecutive points all going up or all going down. If the value of two or more consecutive points is the same, only count the first point and ignore the repeating values. In addition, like values do not make or break a trend.
There you go! We can conclude with statistical confidence that the Michuki Rules had an impact on reducing traffic deaths but subsequent efforts have come to naught.
In the next class, I’ll look more into other rules for interpreting run charts. Have a trendy day (Please clear your tuition fees arrears!)