Hi I am using Linear and exponential forecasting models to do sales forecasting. In the model itself, we use the forecasts of period t to get next forecast and so on. While analyzing the accuracy of the forecast using Mean Absolute Percentage Error, I get good results. But when I compare the intermediate forecast values of the model against the actual time series data, I can see some big Percentage Errors like MAPE might be 15%, but some of the intermediate percentage errors might be as high as 80% and some will be pretty low. So the MAPE averages out to low. I wanted to know 1) Is it wise to compare these intermediate forecasts with actual values? 2) Can we use these high intermediate forecast values to say that forecasts for so and so month might be unreliable?
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
3
MAPE is known to have problems, when the time series have values close to zero. Check whether this is the case, since high MAPEs may be the problem of time series values close to zero, not of model accuracy. For a discussion on accuracy measures I recommend this article by Rob Hyndman and Anne Koehler.
If it is not the problem with MAPE, @Zach advice is spot on, you should always compare the forecasts with actual values, that is how you know how good your forecasts are.
-
$\begingroup$ Well, my forecasts are as expected according to seasonality but the actual values are either high or lower than the forecast. That might be due to randomness in the data or some other factor. This is giving high MAPE for these cases. I am already using deseasonalized data. Should I remove the randomness component as well from the data and then do the forecasting. $\endgroup$– AbhishekCommented May 26, 2011 at 5:07
-
$\begingroup$ @Abhishek, how do you usually remove (or intend to remove in this case) randomness component? $\endgroup$– mpiktasCommented May 26, 2011 at 5:58
-
$\begingroup$ I am not sure about that. I guess that won't be possible since it is randomness which can't be predicted. $\endgroup$– AbhishekCommented May 30, 2011 at 13:06
$\begingroup$
$\endgroup$
Yes, you should absolutely compare your predicted values with actual values. This is good practice with any kind of statistical modeling, not just time series analysis.
If certain months are consistently off, you should use a seasonal model.
