I have 12 months of in-sample data and 12 months of out-of-sample data. I'm trying to calculate the scaled error for an h-step ahead forecast where h=1, 6 and 12. Do I just calculate the error at each of these points and divide each by the average of the in-sample observations, or is there a summation involved? For example, if the out-of-sample values are

1,3,5,3,4,6,7,8,2,1,4,5) and the forecasts are
(2,3,6,3,4,8,6,4,1,7,3,9), the errors are
(-1, 0,-1,0,0,-2,1,4,1,-6,1,-4)
What would be the 6-step-ahead error?


As per definition the 6-month ahead error is the error on the 6th forecast. On your example: $-2$ (you underestimate of 2 units), which correspond to $-2/8 = -25\%$.

Now, you can also have the average error on the next 6 month, which can be computed in various ways (Mean absolute deviation, Mean squared deviation, Mean percent absolute deviation...) to count all the errors with the same sing.

The simplest one is the MAD (Mean Absolute Deviation): compute the the average error after taking their absolute value to make them all positive. In your case $AVG(1, 0, 1, 0, 0, 2) = 4/6 = 66.6%$.

If you want the MAPE, then compute the error in percentage first then take the mean: $AVG(1/1, 0/3, 1/5, 0/3, 0/4, 2/6) = 25.6\%$.

Like me you can prefer to take the percentages relative to the forecast rather than the actuals: $AVG(1/2, 0/3, 1/6, 0/3, 0/4, 2/8) = 15.3\%$

There are may other methods.

• Thank you, but where does the 8 come from in your -2/8? – Angus Jun 28 '18 at 11:18
• If it's not that, then what would it be? – Angus Jun 28 '18 at 11:45
• 8 is the 6th forcast. – AlainD Jun 28 '18 at 13:05
• What is should be is a long long story. I update my answer with the simpest one. – AlainD Jun 28 '18 at 13:09
• Thanks again. I was also looking at cross validation and a rolling forecast as per R. Hyndman. – Angus Jun 28 '18 at 15:36