Is MAPE controversial?

MAPE = Mean Absolute Percent Error.  This is the common error measure used by the Supply Chain profession.  Demand Planners and Demand Forecasters spread across the manufacturing sector use MAPE to measure the performance of their demand forecasts.  This is used cross-sectionally across several items or SKUs for the previous month using a forecast that was created a few months earlier (1, 2, 3 or even 6 month lags are possible).
However, there is raging debate over what to use in the denominator while calculating this sacro-sanct MAPE measure.
Traditional MAPE measure = (Absolute Error)/Actual demand.
Why dont we divide the error by forecast value instead of dividing by Actual sales?  This also introduces a potential forecasting bias.  To reduce MAPE, the planner can actually over-forecast.
Here is what we need to keep in mind:
1.  Whatever we use as denominator, we need to be consistent across the score-card, division, and company as a whole. So the denominator does NOT depends on the level.  It needs to be the same across everything.

2.  Zero demand may result in an ugly “DivbyZero” error in Excel when you use Actuals as the denominator. The practical solution is to set the error percent to a high number such as 9999%. Note the error is not Zero!! Mathematically the error is infinite (but infinity is not a practical supply chain concept).

3.  Setting the individual item error as a high number when demand is zero, should not affect the overall MAPE, since the overall MAPE is NOT a simple average of individual MAPEs.

4.  The denominator problem disappears if in general forecast errors are lower, that is when Forecast is closer to the Actuals; but note that most supply chain issues and demand side biases also disappear when forecast error is smaller!

5. The point is planners/managers/companies should spend less time worrying/arguing about how large the error is when the error exceeds a certain threshold say 50-60%. Although mathematically it matters between 90% and 900%, from a supply chain perspective a 90% error is disastrous! A 900% error is just plain silly and perhaps shows either there is a metrics process flaw or we are chasing after the wrong rat!

6. In summary, we can definitely perform sensitivity analysis on how the division performance will change based on a denominator change. In some instances we have even used the mid-point between actual and forecast. There is a potential to underforecast when the denominator is Actuals.  So you can use a mid-point to what is called as the Symmetric MAPE.

7. Calculating with reference to the Budget is a good practice to measure the attainment of sales goals with reference to the plan targets – so using Budget as the denominator makes sense for measuring sales force performance.  However, note that this is not absolute error in the sense of MAPE and we are answering a different question. So we should stay away from this confusion.