Is WMAPE Over-Rated As An Organizational KPI?
Many are taking shots at the very thought of measuring Forecasts itself leave alone with WMAPE. And some are taking shots at the discipline of Forecasting itself. If forecasting is a waste of time – why businesses and companies spend tons of time forecasting something or other?
In most cases, we forecast volume or unit demand or a derivative – revenues, dollars, weighted average prices, Budgets and Earnings per share. Wall Street rewards companies that forecast better versus companies that do poorly. And of course, they measure the forecasts as well with a version of MAPE – estimated vs. actual EPS.
Those taking pot-shots at forecasting and forecast measurements may have their own motivations. Most have something to do with a version of planning that does not advocate the use of forecasts or demand plans in the supply chain.
If not forecasts then what do they use?
Just an average of the order flow………. concluding that just a plain dumb average is better than the demand plan.
They may have some valid reasons for this:
1. Forecast bias can be uncontrollable and this may lead to measured forecast errors somewhat worse than just using a dumb average.
2. An ill-defined process may have made forecasts unusable – unscientifically splitting the forecasts from Monthly to weekly to daily and forcing this on an MRP process.
3. A revenue driven forecast may have been the primary driver of the unit demand plan with its consequent ills.
If you forecast, then you have to measure.
Are there reasonable alternatives to using a forecast measurement?
This question leads to a stimulating dialogue assessing various perspectives to comprehend the significance of forecast and its accuracy.
First, it is essential to understand that the focus should not be only on the result of the forecast insights but also the process in itself. Many a time, a human planner identifies better parameter processing information instead of relying on plain calculations that result in extreme absurdities. “Deep and trustful collaboration” could negate the problem of the said absurdities – effective exchange of relevant information between different players in Sales, Marketing, Finance and Demand Planning help improve the target that we are want to achieve.
There still remain concerns regarding the nature of optimization of isolated KPIs (Key Performance Indicators) as a cause for a sub-optimal supply chain. Typically companies calculate the total performance as an aggregation of individual KPI. But individual optimization does not necessarily guarantee that the supply chain is at its full potential.
So we need individual measurements as well as measurements that give you insights into the overall performance of the supply chain and the company.
WMAPE, when measured properly along with its peer metrics of Bias and SKU-Mix Error, will lead one in that direction. WMAPE is a composite of both Forecast Bias as well as Mix-level errors. If measured using a price and volume weighted revenue forecast, this will almost always align the company performance with most of the demand side metrics as well as supply-side metrics.
In my opinion, WMAPE in all its glory is not overrated – it is merely misunderstood and misinterpreted.
Its significance lies in the diligent process of collaboration with complementary KPIs that include Bias and Mix-Error along with Order Fill Rates, ROI and Asset Turns.
Several of our Forum users have different views on this subject – please visit our linked-in group to see various comments and post your own comments.
https://www.linkedin.com/groups/1808515/1808515-6420996199267991556.
You can see related Blog entries on WMAPE, Bias and Forecast Accuracy: