3
$\begingroup$

There are a number of posts on Cross-Validated about seasonality in time-series and detrending a dataset, in the context of classical time series models like AR, MA, ARIMA, etc. But my question was more of a question about intuition and practice. As an example, lets use the common airline passenger dataset from this post

So there are different was to deal with the presence of seasonality in a dataset. In the image above there is a clear linear trend and a seasonal trend. The linear trend makes the dataset non-stationary, so we could remove it by just applying linear regression on the time index and then differencing the linear trend from the original time series. This would leave us with something that looks very periodic.

If I start with some simple AR model like:

$$ y_t = \beta_0 + \beta_1*y_{t-1} + \epsilon_t \sim N(0, \sigma^2) $$

To deal with the periodicity I could.

  1. add lags to an AR or ARIMA model that correspond to the period of the seasonality. Since airline data has monthly observations, I could use a yearly or 12 month lag. In that case, the model would become:

$$ y_t = \beta_0 + \beta_1*y_{t-1} + \beta_{2}*y_{t-12}+ \epsilon_t \sim N(0, \sigma^2) $$

  1. The other approach is to just detrend the seasonality before applying the model. So I could use a Fourier transform (or any other set of basis functions) to extract out the form of the seasonal component, then difference out this seasonal component, and then run an AR model such as the original model to predict future values. Or at the least I might have a lower order lag term if some seasonality still persists?

Now practically speaking I could try either approach on a training and test set and see which one performs the best. But from a mathematical or numerical standpoint, I was just trying to understand whether there was one approach that made more sense or less sense? I mean from a numerical standpoint does one approach or the other produce better theoretical results or guarantees? Most AR models are fit with simple linear regression, but complex models like ARMA, or ARIMA models are fit with Kalman filters, so I was not sure whether approach #1 or #2 worked any better from a numerical standpoint?

Any thoughts would be appreciated.

$\endgroup$
1
$\begingroup$

The are several approaches on how to deal with seasonality in time series. You can either model it or remove it and model the residual series. Some popular approaches:

  1. Decomposition: the original time series is decomposed into several components (additive or multiplicative) which are modelled and forecasted individually. Exponential smoothing is a very popular method, capable of also modelling seasonality (Holt-Winters' seasonal method)
  2. Seasonal ARIMA models: Stationarity is a necessary condition before applying ARIMA. Usually, time series with trend and/or seasonal component are non-stationary. Essentially here seasonality is removed via seasonal differencing
  3. Harmonic Regression: Here you use Fourier terms to model the time series. I believe that empirical results have shown that this performs better for long-term components, while performance degrades as the frequency increases.
  4. Dummy variables: Calendar effects can be modelled with indicator variables.
  5. Ignore: You can just ignore the seasonality and directly model the time series. This is sometimes applied in machine learning literature, but inputs usually include historical lags of the target series.

This list is non-exhaustive. The referenced book offers a good starting point in general. Also, note that there is a difference between additive and multiplicative seasonality. I believe that the passenger dataset contains multiplicative seasonal component, because each cycle seems to get more volatile.

$\endgroup$
4
  • $\begingroup$ This does not seem to answer the question. $\endgroup$ – Richard Hardy Sep 19 '20 at 9:03
  • $\begingroup$ @Akylas thanks for the input. Nice summary of different approaches to seasonality in time series prediction. My original question was really about understanding numerically where one approach works versus another approach not working. That is the main question. Like when does using lags in a model work better than simple detrending using Fourier transform. Or when does harmonic regression work better than decomposition. $\endgroup$ – krishnab Sep 19 '20 at 19:14
  • $\begingroup$ Seems like there should be some numerical reason for the difference in performance other than just test all of the models and see which gives you the best output--since that is very prone to overfitting on one dataset, even if you have a holdout training set. $\endgroup$ – krishnab Sep 19 '20 at 19:14
  • $\begingroup$ Note that the one does not exclude the other, you can detrend using Fourier terms and still use lags in order to further model any autocorrelations. From a theoretical standpoint, you would like your in-sample residuals to be uncorrelated, however, there is no guarantee regarding the predictive performance. $\endgroup$ – Akylas Stratigakos Sep 20 '20 at 8:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.