I have 12 months of in-sample data and 12 months of out-of-sample data. I'm trying to calculate the scaled error for an h-step ahead forecast where h=1, 6 and 12. Do I just calculate the error at each of these points and divide each by the average of the in-sample observations, or is there a summation involved? For example, if the out-of-sample values are

1,3,5,3,4,6,7,8,2,1,4,5) and the forecasts are
(2,3,6,3,4,8,6,4,1,7,3,9), the errors are
(-1, 0,-1,0,0,-2,1,4,1,-6,1,-4)
What would be the 6-step-ahead error?


As per definition the 6-month ahead error is the error on the 6th forecast. On your example: $-2$ (you underestimate of 2 units), which correspond to $-2/8 = -25\%$.

Now, you can also have the average error on the next 6 month, which can be computed in various ways (Mean absolute deviation, Mean squared deviation, Mean percent absolute deviation...) to count all the errors with the same sing.

The simplest one is the MAD (Mean Absolute Deviation): compute the the average error after taking their absolute value to make them all positive. In your case $AVG(1, 0, 1, 0, 0, 2) = 4/6 = 66.6%$.

If you want the MAPE, then compute the error in percentage first then take the mean: $AVG(1/1, 0/3, 1/5, 0/3, 0/4, 2/6) = 25.6\%$.

Like me you can prefer to take the percentages relative to the forecast rather than the actuals: $AVG(1/2, 0/3, 1/6, 0/3, 0/4, 2/8) = 15.3\%$

There are may other methods.

• Thank you, but where does the 8 come from in your -2/8? Jun 28, 2018 at 11:18
• If it's not that, then what would it be? Jun 28, 2018 at 11:45
• 8 is the 6th forcast. Jun 28, 2018 at 13:05
• What is should be is a long long story. I update my answer with the simpest one. Jun 28, 2018 at 13:09
• Thanks again. I was also looking at cross validation and a rolling forecast as per R. Hyndman. Jun 28, 2018 at 15:36