A control chart on a chemical reactor's temperature, an extruder's output thickness, or a tank's concentration often looks broken. Points sit outside the limits for long stretches, come back in, then wander out again. The process is running normally and the product is fine, but the chart signals every day.
The usual cause is autocorrelation: consecutive measurements are not independent. This breaks an assumption the individuals chart depends on, and the failure mode is specific and predictable.
What autocorrelation means here
Two measurements are autocorrelated when knowing one tells you something about the next. In a continuous process this is normal physics. A reactor at 84°C now will be near 84°C in five minutes, because thermal mass, feed rates, and control loops all carry state forward. A film coming off an extruder at 0.42 mm now will be near 0.42 mm a metre later.
The strength is measured by the lag-1 autocorrelation coefficient, φ, ranging from −1 to +1. Values above about 0.3 start to cause trouble on a Shewhart chart; values above 0.7 make it unusable.
The individuals chart assumes each observation is an independent draw from the process distribution. When they're not, the moving range no longer estimates what the chart needs it to estimate.
Why the limits come out too tight
The moving range measures the difference between consecutive observations. If consecutive observations are correlated, they're closer together than two independent draws would be, so the moving ranges are small.
For an AR(1) process with lag-1 autocorrelation φ and true process standard deviation σ, the expected moving range is proportional to σ√(2(1 − φ)) rather than the σ√2 you'd get from independent observations. The moving-range estimator therefore returns
σ̂ ≈ σ · √(1 − φ)
So:
Lag-1 autocorrelation φ | σ̂ as a fraction of true σ | Effective limit width |
|---|---|---|
0.0 | 1.00 | ±3.0σ (correct) |
0.3 | 0.84 | ±2.5σ |
0.5 | 0.71 | ±2.1σ |
0.7 | 0.55 | ±1.6σ |
0.9 | 0.32 | ±0.95σ |
At φ = 0.7, your three-sigma limits are effectively sitting at 1.6 sigma. On a perfectly stable process, roughly 11% of points fall outside 1.6 sigma, so you'd expect a false alarm about every nine points.
At φ = 0.9, the limits are at about one sigma, and a third of all points will fall outside them.
Why the pattern looks like runs, not scatter
Tight limits alone would produce scattered points outside on both sides. Autocorrelation produces something more distinctive: the process wanders slowly, drifting above the center line for a stretch and below it for another.
That's exactly what correlated data does. Each value is close to the last, so the series moves smoothly rather than jumping around the mean. Combined with limits that are too tight, you get extended excursions outside the limits followed by extended periods inside, and long runs on one side of the center line.
If your chart shows that pattern, a wandering line rather than random scatter, with runs of ten or twenty points on one side, autocorrelation is the first thing to check.
How to check
Compute the lag-1 autocorrelation. Correlate the series against itself shifted by one position. In a spreadsheet: =CORREL(A2:A100, A3:A101). Above 0.3, you have a problem worth addressing.
Compare the two sigma estimates. Compute the overall standard deviation of all the data and compare it to MR̄/1.128. For independent data these should be similar. For positively autocorrelated data, the moving-range estimate will be substantially smaller. A ratio of overall-SD to MR-based-sigma above about 1.3 is a strong indicator.
This is the reverse of the pattern you see with a process shift, where the overall SD is larger because it absorbed the shift. Both produce the same ratio direction, so check the plot: a shift looks like a step, autocorrelation looks like a wander.
Look at the plot shape. Smooth wandering means correlation. Random jumping means independence.
Fix 1: Sample less often
The simplest and usually the best fix. Autocorrelation decays with time. If temperature readings five minutes apart have φ = 0.85, readings an hour apart may have φ = 0.1.
Find the interval at which the autocorrelation drops below about 0.2 and sample at that interval. Compute the lag-1 correlation at several candidate intervals from your existing high-frequency data to choose.
The cost is slower detection, since you have fewer points per hour. But a chart that signals honestly every two hours beats one that signals meaninglessly every ten minutes.
This fix also has a conceptual justification: if consecutive readings are nearly identical, the extra readings weren't carrying information anyway.
Fix 2: Chart the residuals from a time-series model
Fit an ARIMA model to the series, then chart the residuals (the part the model couldn't predict). If the model is adequate, the residuals are independent and a standard individuals chart on them behaves correctly.
Advantages: uses all the data at the original frequency, and it's the statistically correct approach.
Costs: requires someone who can fit and validate a time-series model, and requires refitting when the process changes. The chart is also harder to explain: a point outside the limits means the value differed from what the model predicted, not that the value was unusual in absolute terms. Operators need training to read it.
This is standard practice in some process industries and overkill in most others.
Fix 3: Chart batch means
Average the readings within each hour or shift and chart the averages. Averaging reduces both the autocorrelation and the variance, and the resulting series is usually much closer to independent.
Advantages: simple, keeps the chart on the original scale, easy to explain.
Costs: slower detection, and a brief excursion within an averaging window gets diluted.
This is the practical middle ground between fixes 1 and 2 and is often the right answer for a chart that a plant reviews shift by shift.
Fix 4: Widen the limits deliberately
Some practitioners compute limits using the overall standard deviation rather than MR̄/1.128 when the data are known to be autocorrelated, on the grounds that the overall SD captures the process's actual wandering.
This works in the narrow sense that the chart stops false-alarming, but it abandons the within-process sigma estimate that makes a control chart a control chart, and it will also hide real shifts. If you do this, document it explicitly as a deliberate deviation, and understand that what you now have is closer to a run chart with limits than a Shewhart chart.
A related case: sampling faster than the process changes
The same problem appears without a continuous process. If you measure five consecutive parts from a single-cavity mold produced seconds apart, those parts share every input condition and their measurements are effectively correlated. The moving ranges measure gauge noise rather than process variation, sigma comes out tiny, and the limits are absurdly tight.
The fix is the same: space the observations out until consecutive values are genuinely independent.
What autocorrelation is not
It isn't a data quality problem, and it isn't something to correct away. A reactor whose temperature is correlated over five minutes is behaving exactly as a reactor should. The chart is the thing that doesn't fit, not the process.
It also isn't the same as a trend. A trend is a sustained directional movement caused by something accumulating; autocorrelation is a tendency for adjacent values to be similar while the series has no net direction. Autocorrelated data will show apparent trends, which is one of the reasons the trend rule misfires on these charts.
In practice
Before concluding a chart is broken, check the lag-1 correlation and compare the two sigma estimates. An I-MR chart maker that reports MR̄ and the derived sigma as numbers makes the second check quick: compute the overall standard deviation separately and look at the ratio. If the MR-based sigma is much smaller and the plot wanders rather than jumps, resample at a longer interval and rebuild the chart. That usually resolves it without any modelling.
Summary
Autocorrelation makes consecutive observations similar, which shrinks the moving ranges, which makes the control limits too tight. At a lag-1 correlation of 0.7, three-sigma limits effectively sit at 1.6 sigma and roughly one point in nine signals on a stable process. The pattern is distinctive: a smoothly wandering line with long runs, not random scatter. Diagnose by computing the lag-1 correlation and comparing the moving-range sigma against the overall standard deviation. The simplest fix is to sample less frequently until consecutive values are independent; charting batch means or time-series residuals are the alternatives.