0
$\begingroup$

My company uses an inventory model (call it IM) to order parts. I have two years of data (Jan 2016 to Dec 2017) that show how many parts were actually ordered and as well as a forecast from Jan-Dec 2017 that shows how many parts the IM model said to order.

I calculated the MASE for the (a) IM Model, (b) Mean, and (c) Naive forecasts using Jan-Dec 2016 as my training set and Jan-Dec 2017 as my test set.

For over 6000 parts, there were more parts that showed that the Mean and Naive forecasts had MASE values less than 1 than the IM model, so I would conclude from that that the IM model was not very good when compared to a Mean or Naive forecast. The problem is, when I apply a Shapiro-Wilks test or a Doornik-Hansen test to the residuals for the period Jan to Dec 2017, there are more parts where normality is satisfied for the IM model than for the Mean and Naive forecasts, which leaves me with two different answers? Could someone please interpret what I'm missing?

$\endgroup$
2
  • $\begingroup$ Two different answers to two different questions -- no paradox here. Decide what you are after and then use the corresponding evaluation. $\endgroup$ Mar 9, 2018 at 13:49
  • $\begingroup$ Does not normality of residuals for more parts tell me that the IM model is better at forecasting than Mean or Naive? $\endgroup$
    – Angus
    Mar 9, 2018 at 13:56

1 Answer 1

2
$\begingroup$

Two different answers to two different questions:

  1. Which forecast gives me the lowest MASE? and
  2. Which forecast gives me normal errors?

-- no paradox so far.

As a forecaster you must have some evaluation criterion for the forecasts, e.g. mean absolute error (MAE). Or perhaps you want to penalize large errors more, then use mean square error (MSE). Or perhaps you care about underpredicting more than overpredicting, then use some asymmetric loss function. For these criteria normality is of no direct concern; it may only kick in indirectly through the shape of the evaluation criterion. In any case, you should justify your choice by the loss function you are facing.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.