# How is ARMA/ARIMA related to mixed effects modeling?

In time series analyses, I have used multi-level or random/mixed effects to deal with auto-correlation issues (i.e., observations are clustered within individuals over time) and added controls are added for some specification of time and for shocks of interest. ARMA/ARIMA seem designed to address similar issues.

The resources I've found online discuss either ARMA/ARIMA or mixed effect models but I don't understand the relationship between the two. Might one want to use ARMA/ARIMA from within a multilevel model? Is there a sense in which the two are equivalent or redundant?

Answers or pointers to resources that discuss this would be great.

-

I think the simplest way to look at it is to note that ARMA and similar models are designed to do different things than multi-level models, and use different data.

Time series analysis usually has long time series (possibly of hundreds or even thousands of time points) and the primary goal is to look at how a single variable changes over time. There are sophisticated methods to deal with many problems - not just autocorrelation, but seasonality and other periodic changes and so on.

Multilevel models are extensions from regression. They usually have relatively few time points (although they can have many) and the primary goal is to examine the relationship between a dependent variable and several independent variables. These models are not as good at dealing with complex relationships between a variable and time, partly because they usually have fewer time points (it's hard to look at seasonality if you don't have multiple data for each season).

-
:Peter Very nice summary. I would only add that time series data is not usually "long" when dealing with weekly/monthly/annual data BUT can get long when dealing with daily/hourly/second data. –  IrishStat Oct 26 '11 at 12:53
Your explanation is quite good, in practice, though I would add a slight caveat. ARIMA models can be implemented as State Space models (R's arima does this, under the hood), also known as Dynamic Linear models (DLMs). DLMs are also extensions from regression (in a different way than Mixed Effects), so I'd guess that there is a deep-down relationship between ARIMA and Mixed-effect models. That doesn't change the differences in practice, which you summarize well. –  Wayne Oct 26 '11 at 23:16
This is very useful. I will point that out that adding a moving average to a multi-level model is certainly possible (and, in the simplest form, is done all the time by adding lagged variables (e.g., the dependent variable at $t-1$). –  Benjamin Mako Hill Oct 27 '11 at 19:26
Benjamin: The whole idea of statistics is to IDENTIFY STRUCTURE not assume it. –  IrishStat Oct 27 '11 at 22:51
I think a complete answer might also mention the difference between time series and panel data. If I understand correctly, ARIMA and similar are primarily used for data where each observation is of the same variable over time. In the multilevel model for change, we are usually focused on panel data and we are modeling a variable measured across a range of individuals, groups, countries, etc, over time. Right? –  Benjamin Mako Hill Jul 21 at 13:00

ARMA/ARIMA are univariate models that optimize how to use the past of a single series to predict that single series. One can augment these models with empirically identified Intervention Variables such as Pulses, Level Shifts , Seasonal Pulses and Local Time Trends BUT they are still fundamentally non-causal as no user-suggested input series are in place. The multivariate extension of these models is call XARMAX or more generally Transfer Function Models which use PDL/ADL structures on the inputs and employ any needed ARMA/ARIMA structure on the remainder. These models can also be robustified by incorporating empirically identifiable deterministic inputs. Thus both of these models can be considered Applications to longitudinal (repeated measures) data. Now the Wikipedia article on multi-level models refers to their application to time series/longitudinal data by assuming certain primitive/trivial i.e. non-analytical structures like "The simplest models assume that the effect of time is linear. Polynomial models can be specified to allow for quadratic or cubic effects of time".

One can extend the Transfer Function model to cover multiple groups thus evolving to Pooled Cross-section time series analysis where the appropriate structure (lags/leads) can be used in conjunction with ARIMA structure to form both local models and an overall model.

-
Multi-level models can also use a general specification for time which add dummies for each time that will capture the average effect for that time period. –  Benjamin Mako Hill Oct 27 '11 at 19:22
:Benjamin The problem with rhat is you are assuming that the seasonality is deterministic and to top it off that the seasonal coefficients are invariant over time as compared to a seasonal pulse one of the ISI-1 dummies which had no effect for the first k time periods but did so after. Another equally possible seasonal structure is the seasonal ARIMA component which uses an adaptive response to prior seasons as compared to your suggested FIXED response. –  IrishStat Oct 27 '11 at 22:44