I made some predictions on some time series data. The plots look good, the predictions line up with the original values quite well. But the error values don't make much sense to me. I calculated the RMSE and the MAE using scikit learn and found that both values tend to be slightly below the mean of the range.

This is consistent through out the results. Another interesting observation was that when the data was constant (and my forecast all over the place) the RMSE and MAE became the constant value i.e. the mean of the range.

This seems to occur to all my predictions which involve different time series.

From reading about the RMSE I leant that it calculated the standard deviation between the predictions and the data. So from the results all my predictions should be off by the average value in the data but the plots seem quite good.

An example of the data being constant and the error being the same, data is always 7 and the RMSE and MAE are also 7.

enter image description here

An example of a (visually) good fit with the error the same as the mean, here the mean of the range is (102000 + 99500)/2 = 100750 and the error is 100715.

enter image description here

I don't understand how these error values help explain the model's performance if all they do is tell me the mean of the range the data was in. The RMSE and MAE values seem quite bad even though the plots seem accurate.

  • 1
    $\begingroup$ Could you share how did you calculated those errors? Ideally including small data example and predictions? This seemed like some issue in calculation to me, it does not make sense that the error should be the mean the range when predictions line up with the original (especially in the second case). $\endgroup$ – Nerxis Apr 21 at 15:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.