Measuring Forecast Error – Constrained or Unconstrained Forecast?

I had a long conversation recently with the President of a CPG company on adding some form of Forecast quality as a metric for his Sales Managers. He was interested in designing a matrix of measures along with quota attainment, promotional performance and account profitability.

What measures to consider?

WMAPE, Bias, Simple attainment all came up.  WMAPE was a tough one since the sales force has to be educated on the meaning and the method behind the madness.  Mad/Mean ratio sounded realistic but it was too mean to call the WMAPE as a Mad mean measure.  Instead of Bias we decided to use the simple attainment as this was something that they all understood. However this will be different from the Quota attainment as the denominator would be different things.

Quota Attainment = Actual Sales / Quota from the Annual Sales Plan or the Budget

Simple attainment = Actual Sales / Rolling forecast from the Demand Plan.  This will be measured with a two month lag.

Overall SKU-mix issues, we settled on the WMAPE and a process to educate the organization on the effectiveness of Weighted Mean Absolute Percent Error.

How do we measure the Rolling Forecast?

The key question is how do we measure the rolling forecast.

They often have supply shortages because of imported products with a long lead time. Since this affects the actual shipments to the customers, the main argument was that the company should measure a constrained version of the forecast that adjusts for the expected shorts and measure them against the actual shipments.

Constrained or Unconstrained? Seems counter-intuitive but the Sales Force either vehemently opposes the metric or wants the metric to be adjusted for issues that they don’t control.

Constrained or Unconstrained

Is it better to correct the actual sales for supply constraints and come up with a version of the actual demand that is unconstrained?

Measurement is an ex-post event that should not affect the actual forecasting process.  If the Sales force is encouraged to think about a constrained forecast, the organization will lose visibility to the true unconstrained demand.

One option will be to use a disciplined process to measure true demand every month and evaluate the unconstrained forecast with this version of true demand.

True Demand = Actual Customer Orders – adjusted for orders intended for future delivery – duplicate orders that exaggerate customer demand in supply shortage situations.

Going another way,

True Demand = Invoiced shipments + missed orders because of supply shortage.

Here is a cross-section of comments received from our Linked-In Group members.

https://www.linkedin.com/groups/1808515/1808515-6269574295412838403.

Please visit http://demandplanning.net/consulting.htm to know about our services offerings at DPLLC.

Follow Us to keep updated on our regular content in the SCM space –

Facebook | Twitter | Instagram | Youtube | Linkedin