What To Measure or Forecast Accuracy – SKU/DC, SKU/National, SKU/Plant?
It is now well settled that we want to use the Weighted MAPE to measure the quality of the demand plan.
This can be done in a portfolio of measures –
1. Weighted MAPE at the SKU/DC level to measure distribution planning
2. Forecast Bias to evaluate persistent forecast bias
3. Some measure of SKU-Mix Error
Although most demand planners and COE professionals understand the mechanics of these measures, there is some confusion on what to measure and why.
This all depends on what you are trying to drive – what you are using the forecast for? So level of aggregations matter as well.
Some simple rules:
1. SKU sold and shipped in many DCs but made in a single plant – the Plant cares about the quality of the consolidated plan.
2. SKU/Customer level measure will be helpful to evaluate the quality of the promotional forecasts and Sales Manager’s performance.
3. A common Raw Material is used in producing multiple SKUs. Potato is used in making chips, French Fries, Potato pies with the Potato as the raw material. Procurement cares about the quality of the Potato plan. This is derived from the FG demand but consolidated to the Ingredient level. So measure a long-term horizon at an aggregated level.
Doing the above will need other nuances – you need to have the hierarchy defined in such a way that your demand could be aggregated to the raw material level.
4. A common resource is used to produce multiple products. To evaluate the capacity for this resource, the capacity planner cares about the quality of the long-term forecast aggregated to the resource level.
You can list a variety of examples and circumstances – but it is important to define what the demand plan is expected to solve and who is using this. Then we define the measure appropriately.
Please visit http://demandplanning.net/consulting.htm to know about our services offerings at DPLLC.
Follow Us to keep updated on our regular content in the SCM space –
Hi Mark,
Sorry I did not see this comment – typically the blog is inundated with a lot of spam so it is difficult to filter this out.
Your points are well taken. Tracking signal is an APICS measure that uses the Statistical control theory and it incorporates the significance into the calculation. So if your tracking signal is more than 3.75 in absolute value, this indicates a persistent bias. I would use at least six months worth of calculations.
What I have referred to is for cross-sectional performance across SKUs. So for inventory planning you may want to calculate this at the item level of family level. We use anywhere from six to nine months and use a T-stat to flag bias.
We can talk more if you are interested. We have an outsourced service that will provide scorecards on a monthly basis that can be analyzed in any DV tool.
Mark
Hi Dr. Chockalingum,
I noticed that you do not mention use of the Tracking Signal (i.e. https://en.wikipedia.org/wiki/Tracking_signal) to measure forecast bias.
I’ve been looking into it and I don’t particularly like some things about the Tracking Signal. For instance, when there is zero forecast/zero actuals in a period, it can make a significant change in the tracking signal. Also, I can’t find any reference to how many periods should be used…all the examples start with a period 1 and go to period 3, or 5, or 12, and they always accumulate the data. At some point bias in the past forecasts must be considered irrelevant so it seems the selected periods should be “rolling.”
Your publications use simply the forecast error to illustrate bias, but how to put that into practice? How to accumulate the data, and at what point does one say a forecast has “triggered” a bias switch?
Our company has about 2000 sku’s and we’re looking for ways to detect bias on a monthly basis for review purposes.
Thank you!
Mark J.