Why am I getting different responses from MASE calculations and normality of residuals?

Question

My company uses an inventory model (call it IM) to order parts. I have two years of data (Jan 2016 to Dec 2017) that show how many parts were actually ordered and as well as a forecast from Jan-Dec 2017 that shows how many parts the IM model said to order.

I calculated the MASE for the (a) IM Model, (b) Mean, and (c) Naive forecasts using Jan-Dec 2016 as my training set and Jan-Dec 2017 as my test set.

For over 6000 parts, there were more parts that showed that the Mean and Naive forecasts had MASE values less than 1 than the IM model, so I would conclude from that that the IM model was not very good when compared to a Mean or Naive forecast. The problem is, when I apply a Shapiro-Wilks test or a Doornik-Hansen test to the residuals for the period Jan to Dec 2017, there are more parts where normality is satisfied for the IM model than for the Mean and Naive forecasts, which leaves me with two different answers? Could someone please interpret what I'm missing?

Answer 1

Two different answers to two different questions:

1. Which forecast gives me the lowest MASE? and
2. Which forecast gives me normal errors?

-- no paradox so far.

As a forecaster you must have some evaluation criterion for the forecasts, e.g. mean absolute error (MAE). Or perhaps you want to penalize large errors more, then use mean square error (MSE). Or perhaps you care about underpredicting more than overpredicting, then use some asymmetric loss function. For these criteria normality is of no direct concern; it may only kick in indirectly through the shape of the evaluation criterion. In any case, you should justify your choice by the loss function you are facing.