12
$\begingroup$

What are the differences in using Generalized Linear Models, such as Automatic Relevance Determination (ARD) and Ridge regression, versus Time series models like Box-Jenkins (ARIMA) or Exponential smoothing for forecasting? Are there any rules of thumb on when to use GLM and when to use Time Series?

$\endgroup$
2
  • 2
    $\begingroup$ Ridge regression is not a generalized linear model. The addition of the $\mathcal{L}_2$ penalty makes it a minimax estimator. It is a modification of a GLM. In general, however, GLMs do not use make use of autoregressive covariance structures, but may include lagged fixed effects. $\endgroup$
    – AdamO
    Commented Jan 29, 2018 at 20:58
  • $\begingroup$ GLM does not forecast trends, seasonality, and cycles. ARIMA does. $\endgroup$
    – henryjhu
    Commented Apr 27, 2020 at 23:27

2 Answers 2

6
$\begingroup$

Not really an expert but this question has been unanswered for a while, so I will try an answer: I can think of 3 differences between GLMs and Time series models a là Box and Jenkins:

1) GLMs are rather to model variable Y as function of some other variable X (Y = f(X) ). In the time series models you are (mostly?) modeling variable Y as function of itself, but from previous time steps ( Y(t) = f(Y(t-1), Y(t-2),...) );

2) Related to the previous point: GLMs do not consider per se autocorrelation of the input covariate, while the time series models like ARIMA are auto-correlative in nature;

3) I think the auto-regressive models base on the assumption that residuals are normal with zero mean, whereas GLMs accept more complex data structure of the response variable, possibly having a non-normal distribution (Gamma, Poisson, etc).

Are there any rules when to use GLM and when to use Time Series? Unless you are considering in your model time as a random effect, I think GLMs are simply the wrong approach to model time series.

$\endgroup$
3
  • $\begingroup$ Your comment 1) is not at all correct, Time Series Models ( Box & Jenkins models) include ARMAX models a.k.a. Transfer Function Models which can include input (predictor series) that can use user-specified predictors and latent deterministic structure ( like pulses, step/level shifts, seasonal pulses local time trends) waiting to be identified . See stats.stackexchange.com/search?q=user%3A3382+transfer+Function+ for more discussions $\endgroup$
    – IrishStat
    Commented Nov 25, 2018 at 21:19
  • 1
    $\begingroup$ This comment is not all correct. General linear model can account for auto-correlation in the error terms. $\endgroup$
    – lzstat
    Commented Nov 28, 2019 at 1:51
  • $\begingroup$ lzstat GLM = generalized linear model != general linear model. The answer is basically correct, even though there are hybrids. $\endgroup$
    – jacques
    Commented May 24, 2021 at 20:25
1
$\begingroup$

I myself was studying neural behavior a long time and I must say that GLMs did a really good job in predicting the complex neural behavior based on external factors but also the activity of other neurons or the past of that specific neuron (i.e. refactory period, rhytmic modulations etc.). I also used them for extensive surrogate modelling, studying zero-inflation, response behavior, noise correlations and more.

Today, I am often surprised that people usually do not even really know what GLMs are but everyone knows AR models. I still did not encounter many situations in which I'd prefer an AR model.

I think there is a common misconception that GLMs do not account for temporal correlations (i.e. auto-correlation or cross-correlations) but that is not true.

My rule of thumb when to use an autoregressive model was: Only when I have to or the nature of the problem suggests! Whenever the main predictor is the past of the predicted quantity itself AR models should be considered. They are based on the past and error terms. For example a pendulum would offer itself for AR models. Things that may be described by a differential equation. I.e. if there are no independent variables that carry information about the future.

If I don't have to I would always prefer GLM models over AR models, because in my perspective they are clearer, more interpretable - at least for me. However, I think the best way to decide is a) really understand your problem and b) try to model it yourself on paper and math. Eventually you may naturally arrive at one or the other model. For proper modelling both variants may need adoptions and modifications. The more complex you model things, the closer both descriptions of your data get. Often the question is, whether you have an appropriate solver for your approach. If you tweak a GLM too much it may not be a convex problem anymore and the likelihood function has local maxima. Sometimes it makes more sense to switch to an EM Algorithm or evolutionary optimizations.

Generally, I believe that the more you get into it, the more confusing are the different terminologies - especially when it comes to crazy variations of GLMs.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.