We are often interested in estimating the limiting distribution of a parameter in situations where the data exhibit dependence within clusters. For example, a study of the effects of a household-level treatment on household-level outcomes must contend with the possibility that households within villages will have correlated outcomes. Thus, when computing standard errors for the treatment effects, we typically take this clustering into account by using a "cluster robust" covariance estimator or perhaps a "random effects" model. (See http://bit.ly/bAah5L for an example.)

The properties of these covariance estimators are typically studied by assuming the source of the clustering are common "shocks" that occur within a group. That is for groups indexed by $g$ and units within groups indexed by $i$, we typically write down a model of the form, $y_{ig} = \alpha_g + \epsilon_{ig}$, where the $\alpha_g$'s denote the group level shocks. We then typically assume that the $\alpha_g$ are independent across groups with group level variance $\sigma^2_{\alpha}$ and the $\epsilon_{ig}$ are independent draws with variance $\sigma^2_{\epsilon}$, in which case the intra-cluster correlation is the familiar $\frac{\sigma^2_{\alpha}}{\sigma^2_{\alpha}+\sigma^2_{\epsilon}}$.

Here's what I am wondering: does this linear "group shocks" model implicitly rule out important forms of within-cluster correlation? (I am limiting the question to correlation for now, and not other types of non-correlation dependence.) That is, are there ways that within-cluster correlation can arise that cannot, through algebraic manipulation, be expressed in terms of a group-level shock? Or is the group level shock characterization general with respect to within group correlation?

I am inclined to try working out some of the examples, but I thought I'd put this up to see if someone else has already done the heavy (or maybe it's not so heavy) lifting.

If this is not a well-posed question (i.e. unanswerable), I'd also welcome comments about why.

(Note I am not putting "clustering" on the tag list here because it's not about cluster methods as the term is usually applied.)


1 Answer 1


The "random shocks" you are describing is just a random intercepts model. Theoretically, random intercepts are one of the simplest and yet most useful models. In application it may be yet another case of spherical cows.

The random intercept framework is a motivating example for, obviously, a correctly specified mixed model, but also generalized least squares or generalized estimating equations using an exchangeable correlation structure. As you have correctly noted, these "shocks" induce a variance-covariance structure that is easily expressed as

$$ \text{cov} (Y_i, Y_j) = \left\{ \begin{array}{ccc} 0 &\text{if} & \text{$i$, $j$ not in same cluster} \\ \rho \sigma^2 &\text{if} & \text{$i \ne j$ in same cluster} \\ \sigma^2 &\text{if} & i = j \\ \end{array} \right.$$

An interesting math-fact is that random intercepts restrict $\rho> 0 $ which applies to most (but not all) situations, it's possible for intracluster correlation to be greater than 1. To express and simulate these data, you can use a QR decomposition of this variance covariance structure and transform independent normal data.

The ICC and the exchangeable correlation structure is well motivated even if the assumptions are not precisely met. It's possible for one to add, layer-by-layer, elements of complexity to these data. You might deal with nested, cross-nested effects which trivially extend these formulations via additional random terms, or covariance cases. Examples are teeth measured over time within the mouth of the same pediatric patient (cross-nested) or student performance over time within a school district (nested if no migration). Whether these meet your "shock" framework is not clear.

Two examples, however, that are not easily expressed in terms of a single additive constant I think will show you how a "shock" framework is not adequate to express all covariance structures.

  • Autoregressive trends: a random slope framework where the random term interacts with time in a repeated measure framework. Under the assumption of balanced design, there are rarely meaningful differences between results from various dependent data designs. Once cluster imbalance is introduces through varying levels of follow-up, the level of dependence within clusters cannot be easily expressed as a single additive constant. Subjects with less follow-up will have more correlation among their scant observations than subjects with greater follow-up.
  • Unmeasured dependence: we may recruit and randomize subjects within community without even knowing they are living in the same household, receiving concordant or discordant treatments with synergistic effects. This trait, called contamination, is a serious and largely unaddressed aspect in clinical trials. It's entirely possible subjects in cancer trials sit in the same infusion centers receiving investigational treatment and sharing their experience of treatment and symptoms with one another.

Not the answer you're looking for? Browse other questions tagged or ask your own question.