# How to interpret MASE for longer horizon forecasts?

After looking at Hyndman and Koehler, 2006 and applying the metric to my own data, I have been convinced that MASE is a better metric for evaluating forecast error than the method I had been previously using (MAPE), at least for short horizon forecasts. However, I am struggling with how to interpret the value of the MASE at longer horizons.

At short horizons it makes sense, it is effectively quantifying how much better (or worse) your forecast did than one of the simplest forecasting methods (the naive/seasonal naive). If your MASE is greater than 1 then, you may want to be suspect that your forecasting method is adding value (assuming there have been no major changes in your data between the in sample and out of sample values).

Where I get confused is for longer horizon forecasts. Imagine I have a situation like below where I have a random walk series I want to forecast (black). For convenience, I will be looking at in sample errors. In cyan, I have plotted the rolling horizon 20 naive forecast for the series as well at the one step (horizon 1) naive forecast in orange which would be used to compute the denominator of the MASE.

The horizon 20 forecast is much worse for obvious reasons, it has less information than the horizon 1 forecast. Accordingly, the MASE reflects this and has a value of 2.88. However, we know that the naive forecast is the best forecast we can make for a random walk series, so it is the forecast we should be using even though the MASE is "bad".

It is easy to know in this circumstance that the MASE being greater than 1 is ok because we have a theoretical grounding, but this will not be the case for most series we want to evaluate. How do we properly interpret the MASE for these longer horizon applications then? Should we create a hMASE which would be a horizonal dependent MASE to match the horizon of interest changing it from

$$MASE = \frac{\sum_{t=1}^{T} \left| e_t \right|}{\frac{T}{T-1}\sum_{t=2}^T \left| Y_t-Y_{t-1}\right|}$$

to

$$hMASE = \frac{\sum_{t=1}^{T} \left| e_t \right|}{\frac{T}{T-h}\sum_{t=h+1}^T \left| Y_t-Y_{t-h}\right|}$$

or would we need to also calculate the MASE for the naive forecast and use that as a comparison? Is there some other method I am not thinking of? What If I am evaluating both short and long term forecasts? Would I include multiple errors for each in sample data point (one for each horizon) in the denominator of my MASE?

In your case, I'd use a naive 20-step-ahead forecast as the benchmark, i.e., your $hMASE$. Just make it clear in your writeup what the benchmark is that you are comparing "your" forecast against.