0
$\begingroup$

I have been checking how each error metric works in the hope to find the best one for my data but it can be quite tricky actually.

I have monthly time series data and I am running a SARIMA model to predict my next 12 data points. Below you can see the chart and the outputs of MAE and RMSE.

MAE: 0.3452288 and RMSE: 0.4714007

I have some questions:

  1. Which metric would make more sense the type of problem I have, MAE or RMSE?
  2. How can I interpret if the values of MAE and RMSE that I got are good or bad in relation to my data?

enter image description here

Improve this question
$\endgroup$
1

2 Answers 2

Reset to default
1
$\begingroup$

I think this is difficult to answer without knowing what you're trying to achieve with the forecasts. For example, what decisions will you be making based on the forecasts? This would dictate what types of errors you would care more or less about (e.g. trading off bias for variance). Note that the RMSE is minimized by the mean and the MAE by the median - what is more meaningful will again depend on what you're trying to achieve.

Improve this answer
$\endgroup$
1
  • $\begingroup$ The idea is to compare two models. One model predicted with SARIMA (the one you see in the post) and another one where machine learning or neural nets are used. The idea is to discover if using traditional time series forecasting is better or not in our case $\endgroup$
    – Rods2292
    Jul 3, 2022 at 17:25
0
$\begingroup$
  1. I think that your question comes from not understanding the interpretation of these error functions. Please, see the link below, I think it will make things more clear for you:

https://www.statology.org/mae-vs-rmse/ https://www.quora.com/What-is-the-interpretation-of-RMSE-MAE-MPE-MAPE-and-MASE-in-testing-forecasting-accuracy

  1. The lower MAE/RMSE the better, these are relative functions which means, you cannot really say they're "good" or "bad" in relation to your data from a certain threshold, but for example compare the quality of forecasts from two models. They are interpretable however, see the links below:

https://www.statology.org/how-to-interpret-rmse/

How do I interpret mean absolute error (MAE) or mean absolute percentage error (MAPE) in layman words?

So as @ergodiclife mentioned, everything depends on what you actually want to achieve.

Improve this answer
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.