Tracking Signal
Tracking signal is a measure used to evalue if the actual demand does not reflect the assumptions in the forecast about the level and perhaps trend in the demand profile. In Statistical Process Control, people study when a process is going out of control and needs intervention.
Similarly Tracking signal tries to flag if there is a persistent tendency for actuals to be higher or lower systematically. If Forecast is consistently lower than the actual demand quantity, then there is persistent under forecasting and Tracking Signal will be positive.
Tracking Signal is calculated as the ratio of Cumulative Error divided by the mean absolute deviation. The cumulative error can be positive or negative, so the TS can be positive or negative as well.
TS should pass a threshold test to be significant. If Tracking Signal > 3.75 then there is persistent under forecasting. On the other hand, if this is less than -3.75 then, there is persistent over-forecasting.
So in essence, |TS| > 3.75 implies a forecast bias ==> TS < -3.75 or TS > 3.75 implies a bias.
So what is magical about 3.75. This is an approximation using the relationship between a normally distributed forecast error and the Mean Absolute deviation.
In General, Forecast Error (using RMSE) * 0.8 = MAD.
At 99% promised service level, you will be using a 3 Sigma level. As a measure of MAD, this translates into 3.75 MAD hence the 3.75 as the threshold for TS.
The Tracking Signal is computed using the following formula –
When a forecast equals an actual value, the numerator for the tracking signal will not change. To add zero to anything will not increase. However, the denominator which is the mean absolute deviation MAD will change. It will become smaller. To average a zero into the numbers will cause MAD to drop. Given this, with perfect forecasts – forecast equal actual – will the Tracking signal not increase?????
I am from Brasi and I repeat the question below:
Hi , regarding TS +/-4, if a situation over that limit occurs only once, should we do anything to correct the forecasting process?
Hi Dr., regarding TS +/-4, if a situation over that limit occurs only once, should we do anything to correct the forecasting process?
Hi
I read your posting well. Because of your posting, I was able to understand the concept of forecast error measure.
I have one question.
How can I deal with the outlier figures when calculating the TS?
Do I have to calculate the TS without the outlier value, otherwise include the outlier value?
Could you tell me the criterion to distinguish outlier and non-outlier?
I’m sorry to ask a lot of questions.
David,
Thank you for the question.
The threshold of 4 is really a threshold of 3.75. This 3.75 number comes from the statistical control limit theory which establishes the relationship between Mean Absolute Deviation and Standard Deviation.
The relationship between MAD and the Standard deviation in a normally distributed population is established as
1.25 MAD = 1 SD (standard deviation of the distribution).
Under normal conditions, Mean +/- 3 Standard deviations should account for 99.7% of the distribution of observations. Only .3% of the observations will be outside the 3SD range. Using the above relationship, this would mean either 3SD or 3.75 MAD.
So when you measure tracking signal, you compare your bias to this threshold to see if your forecast process is out of control either on the positive side or negative side. If you are consistently underforecasting, then your TS will be a positive number. In every period you evaluate the net bias as a ratio of MAD. If the net Bias is over 3.75 * MAD, this is the same as saying that your net cumulate error is beyond 3 Standard deviations from the mean. So this is when you establish a significant bias and start the process to control the error.
The significant bias or the fact that it is out of control could mean that your series is being generated by a different process with a different mean compared to the mean you are publishing as the forecast to the supply chain. The corrective action should result in re-evaluation of the model to control the error and revise your forecast.
So the tracking signal is a definitive call to action when it hits the threshold of 3.75 or above.
Why sometimes is recommended to consider a TS over + – 4 ?
Could you asisst me on it ?