How multi-level models account for correlation and what kinds of correlations? Multi-level models are often shown using figures such as below. These pictures say that observed data at the lowest level (BLUE TRIANGLES) come from some independent populations (BLUE CURVES). But the parameters of these independent populations ($\theta$s), themselves, come from a higher-order population of parameters (GREEN CURVE).
Say I have ~ $7000$ math scores from 160 schools. Here, the scores are the BLUE TRIANGLES assumed to have come from their independent school-level population scores.
At the same time, the $\theta$s of these independent school-level population scores is assumed have come from a higher-order population of $\theta$s.
Question: I can see that schools' $\theta$s is a random effect BUT what kind of correlation (between what?) are we accounting for in this picture?

 A: Multi-level models of this type are accounting for positive correlation among observations within a group (i.e. among students within a school).  We can work this out as follows:

*

*let's say that $x_{ij}$ is the observation for the $j$th student in the $i$th school

*according to the statistical model, $x_{ij} = \mu + \epsilon_{b,i} + \epsilon_{w,j}$ where $b$ stands for "between" (school-level variance) and $w$ stands for "within" (student-level variance)

*the correlation between any two observations is equal to $\sigma_{ij,kl}/(\sigma_{ij} \sigma_{kl})$

*$\sigma_{ij}$ is the same for all observations: $\sigma_{ij}^2= E[(x_{ij}-\mu)^2] = E[\epsilon_{b,i}^2 + 2 \epsilon_{b,i} \epsilon_{w,j} + \epsilon_{w,j}^2]$. Because all of the $\epsilon$ are independent by definition, the middle term disappears and we're left with $\sigma_{ij} = \sqrt{\sigma^2_w+\sigma^2_b}$.

*if $i\neq k$ (i.e. we're looking at students in two different schools) then the covariance $\sigma_{ij,kl}=0$: when we expand out $E[(\epsilon_{b,i} + \epsilon_{w,j})(\epsilon_{b,k} + \epsilon_{w,l})]$ we end up with the sum of expectations of products of independent variables ...

*if $i=k$ (e.g. $\sigma_{ij,il}$) then we have $E[(\epsilon_{b,i} + \epsilon_{w,j})(\epsilon_{b,i} + \epsilon_{w,l})]$; the only term that's left after we expand and cancel is $E[\epsilon_{b,i}^2] = \sigma^2_b$.

So after all this we have a correlation of 0 for students in different schools, and correlation of $\sigma^2_b/(\sigma^2_b + \sigma^2_w)$  for students in the same school. As the between-school variance goes to 0 (i.e. all schools are really identical in their characteristics), correlation goes to 0. As $\sigma^2_b$ gets large relative to $\sigma^2_w$ (there's a lot more difference between schools than between students within a school), correlation goes to 1.
