sarimax Parameter Selection

Question

I'm training a Sarimax model using recent 20 observations sampled monthly, PACF and ACF plots of the series are:

I'm fairly new to time series, but according to tutorials and articles, I've come to an understanding that lags under the confidence intervals are insignificant and lags above it are considered significant.

However, taking large lags tends to overfit the model, so I've chosen my $p,q,P$ and $Q$ range in [0,2], $d, D$ in [0,1] and $m$ in [6,12]. Unfortunately, this choice is not performing well in terms of the MAPE;

Observations:

day                         energy_sum    
2018-07-31 00:00:00+00:00   355.237000  
2018-08-31 00:00:00+00:00   208.775000  
2018-09-30 00:00:00+00:00   481.245999  
2018-10-31 00:00:00+00:00   545.004000  
2018-11-30 00:00:00+00:00   574.898000  
2018-12-31 00:00:00+00:00   527.699000  
2019-01-31 00:00:00+00:00   532.052000  
2019-02-28 00:00:00+00:00   404.393000  
2019-03-31 00:00:00+00:00   501.846000  
2019-04-30 00:00:00+00:00   367.914001  
2019-05-31 00:00:00+00:00   423.271000  
2019-06-30 00:00:00+00:00   465.579000  
2019-07-31 00:00:00+00:00   387.427000  
2019-08-31 00:00:00+00:00   209.631000  
2019-09-30 00:00:00+00:00   446.889000  
2019-10-31 00:00:00+00:00   504.284000  
2019-11-30 00:00:00+00:00   328.485000  
2019-12-31 00:00:00+00:00   299.862000  
2020-01-31 00:00:00+00:00   325.123000  
2020-02-29 00:00:00+00:00   75.571000

Can Anyone suggest how can I improve the performance?

Answer 1

• It makes no sense to fit the season length parameter. For monthly data with yearly seasonality, set $m=12$. Choosing right value of m in SARIMA models. Honestly, with less than multiple years of history (20 months is less than two years!), SARIMA makes little sense, because you can't do the requisite seasonal differencing.

• 20 data points is very little to do forecasting, especially the more complex ARIMA models. Even an ARIMA(1,0,0) model with an intercept fits three parameters, the intercept, the AR1 coefficient and the residual variance. Anything more complex than that or ARIMA(0,0,1) is definitely overfitting. Given that you write you are doing SARIMAX, that implies that you are also estimating one or more parameters for your exogeneous regressors. In such a case, with only 20 observations, I would not do any time series modeling whatsoever.

• Best method for short time-series, because, again, only 20 observations.

• Don't dig into ACF/PACF plots, especially not for short time series. Use an established auto.arima tool, like the one in the forecast package for R. This uses information criteria for model order selection, which is much better than the old Box-Jenkins approach.

• If you have only 20 observations (notice a pattern here?), plus a large amount of residual noise, there is little you can expect in terms of accuracy. How to know that your machine learning problem is hopeless?

• ARIMA models do not aim to minimize the MAPE, they aim at unbiased expectation forecasts. Forecasts that minimize the MAPE are different. If you really want to minimize the MAPE, you can probably get a better result by taking the ARIMA expectation forecast and reducing it a bit. This is really gaming the metric. I have never seen a business decision that would profit from a MAPE-optimal forecast (an MSE-optimal forecast, which is the expectation, is indeed useful). What are the shortcomings of the Mean Absolute Percentage Error (MAPE)?