16
$\begingroup$

In panel data analysis, I have used multi-level models with random/mixed effects to deal with auto-correlation issues (i.e., observations are clustered within individuals over time) with other parameters added to adjust for some specification of time and shocks of interest. ARMA/ARIMA seem designed to address similar issues.

The resources I've found online discuss either times series (ARMA/ARIMA) or mixed effect models but beyond being build on regression, I don't understand the relationship between the two. Might one want to use ARMA/ARIMA from within a multilevel model? Is there a sense in which the two are equivalent or redundant?

Answers or pointers to resources that discuss this would be great.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
13
$\begingroup$

I think the simplest way to look at it is to note that ARMA and similar models are designed to do different things than multi-level models, and use different data.

Time series analysis usually has long time series (possibly of hundreds or even thousands of time points) and the primary goal is to look at how a single variable changes over time. There are sophisticated methods to deal with many problems - not just autocorrelation, but seasonality and other periodic changes and so on.

Multilevel models are extensions from regression. They usually have relatively few time points (although they can have many) and the primary goal is to examine the relationship between a dependent variable and several independent variables. These models are not as good at dealing with complex relationships between a variable and time, partly because they usually have fewer time points (it's hard to look at seasonality if you don't have multiple data for each season).

Improve this answer
$\endgroup$
5
  • 1
    $\begingroup$ :Peter Very nice summary. I would only add that time series data is not usually "long" when dealing with weekly/monthly/annual data BUT can get long when dealing with daily/hourly/second data. $\endgroup$
    – IrishStat
    Commented Oct 26, 2011 at 12:53
  • 3
    $\begingroup$ Your explanation is quite good, in practice, though I would add a slight caveat. ARIMA models can be implemented as State Space models (R's arima does this, under the hood), also known as Dynamic Linear models (DLMs). DLMs are also extensions from regression (in a different way than Mixed Effects), so I'd guess that there is a deep-down relationship between ARIMA and Mixed-effect models. That doesn't change the differences in practice, which you summarize well. $\endgroup$
    – Wayne
    Commented Oct 26, 2011 at 23:16
  • 1
    $\begingroup$ This is very useful. I will point that out that adding a moving average to a multi-level model is certainly possible (and, in the simplest form, is done all the time by adding lagged variables (e.g., the dependent variable at $t-1$). $\endgroup$
    – mako
    Commented Oct 27, 2011 at 19:26
  • $\begingroup$ Benjamin: The whole idea of statistics is to IDENTIFY STRUCTURE not assume it. $\endgroup$
    – IrishStat
    Commented Oct 27, 2011 at 22:51
  • $\begingroup$ I think a complete answer might also mention the difference between time series and panel data. If I understand correctly, ARIMA and similar are primarily used for data where each observation is of the same variable over time. In the multilevel model for change, we are usually focused on panel data and we are modeling a variable measured across a range of individuals, groups, countries, etc, over time. Right? $\endgroup$
    – mako
    Commented Jul 21, 2014 at 13:00
7
$\begingroup$

ARMA/ARIMA are univariate models that optimize how to use the past of a single series to predict that single series. One can augment these models with empirically identified Intervention Variables such as Pulses, Level Shifts , Seasonal Pulses and Local Time Trends BUT they are still fundamentally non-causal as no user-suggested input series are in place. The multivariate extension of these models is call XARMAX or more generally Transfer Function Models which use PDL/ADL structures on the inputs and employ any needed ARMA/ARIMA structure on the remainder. These models can also be robustified by incorporating empirically identifiable deterministic inputs. Thus both of these models can be considered Applications to longitudinal (repeated measures) data. Now the Wikipedia article on multi-level models refers to their application to time series/longitudinal data by assuming certain primitive/trivial i.e. non-analytical structures like "The simplest models assume that the effect of time is linear. Polynomial models can be specified to allow for quadratic or cubic effects of time".

One can extend the Transfer Function model to cover multiple groups thus evolving to Pooled Cross-section time series analysis where the appropriate structure (lags/leads) can be used in conjunction with ARIMA structure to form both local models and an overall model.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Multi-level models can also use a general specification for time which add dummies for each time that will capture the average effect for that time period. $\endgroup$
    – mako
    Commented Oct 27, 2011 at 19:22
  • 1
    $\begingroup$ :Benjamin The problem with rhat is you are assuming that the seasonality is deterministic and to top it off that the seasonal coefficients are invariant over time as compared to a seasonal pulse one of the ISI-1 dummies which had no effect for the first k time periods but did so after. Another equally possible seasonal structure is the seasonal ARIMA component which uses an adaptive response to prior seasons as compared to your suggested FIXED response. $\endgroup$
    – IrishStat
    Commented Oct 27, 2011 at 22:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.