Generalized Linear Models vs Timseries models for forecasting What are the differences in using Generalized Linear Models, such as Automatic Relevance Determination (ARD) and Ridge regression, versus Time series models like Box-Jenkins (ARIMA) or Exponential smoothing for forecasting? Are there any rules of thumb on when to use GLM and when to use Time Series?
 A: Not really an expert but this question has been unanswered for a while, so I will try an answer: I can think of 3 differences between GLMs and Time series models a là Box and Jenkins:
1) GLMs are rather to model variable Y as function of some other variable X (Y = f(X) ). In the time series models you are (mostly?) modeling variable Y as function of itself, but from previous time steps ( Y(t) = f(Y(t-1), Y(t-2),...) );
2) Related to the previous point: GLMs do not consider per se autocorrelation of the input covariate, while the time series models like ARIMA are auto-correlative in nature;
3) I think the auto-regressive models base on the assumption that residuals are normal with zero mean, whereas GLMs accept more complex data structure of the response variable, possibly having a non-normal distribution (Gamma, Poisson, etc).
Are there any rules when to use GLM and when to use Time Series? Unless you are considering in your model time as a random effect, I think GLMs are simply the wrong approach to model time series.
A: I myself was studying neural behavior a long time and I must say that GLMs did a really good job in predicting the complex neural behavior based on external factors but also the activity of other neurons or the past of that specific neuron (i.e. refactory period, rhytmic modulations etc.). I also used them for extensive surrogate modelling, studying zero-inflation, response behavior, noise correlations and more.
Today, I am often surprised that people usually do not even really know what GLMs are but everyone knows AR models. I still did not encounter many situations in which I'd prefer an AR model.
I think there is a common misconception that GLMs do not account for temporal correlations (i.e. auto-correlation or cross-correlations) but that is not true.
My rule of thumb when to use an autoregressive model was: Only when I have to or the nature of the problem suggests! Whenever the main predictor is the past of the predicted quantity itself AR models should be considered. They are based on the past and error terms. For example a pendulum would offer itself for AR models. Things that may be described by a differential equation.
I.e. if there are no independent variables that carry information about the future.
If I don't have to I would always prefer GLM models over AR models, because in my perspective they are clearer, more interpretable - at least for me.
However, I think the best way to decide is a) really understand your problem and b) try to model it yourself on paper and math. Eventually you may naturally arrive at one or the other model. For proper modelling both variants may need adoptions and modifications. The more complex you model things, the closer both descriptions of your data get.
Often the question is, whether you have an appropriate solver for your approach. If you tweak a GLM too much it may not be a convex problem anymore and the likelihood function has local maxima. Sometimes it makes more sense to switch to an EM Algorithm or evolutionary optimizations.
Generally, I believe that the more you get into it, the more confusing are the different terminologies - especially when it comes to crazy variations of GLMs.
