2
$\begingroup$

Suppose we have population P that is split into large number of close size sub-populations (e.g. country is split to zips). For each sub-population a small sample was drawn and average of a variable X was calculated.

The researcher has only access to the average values and he/she needs to study a correlation between X and a random variable Y that researcher has the full access to the population data.

What is the correct way to organize the correlation calculation?

Concern #1 is that the researcher doesn't have access to the individual instances used for the sampling of Y. So if he/she take a ratio between the mean of X and mean of Y it might not be the same as the mean of the ratio X/Y.

Concern #2 is the small size of the X samples. However, since the number of sub-populations is large the same forces that lead to the CLT will be applied to the correlation.

$\endgroup$
1
  • 2
    $\begingroup$ (+1) Make sure you first understand the ecological fallacy. This suggests that, without additional knowledge or assumptions, this effort might be doomed. The CLT will not rescue it. $\endgroup$
    – whuber
    Oct 24, 2014 at 16:04

1 Answer 1

0
$\begingroup$

This isn't an answer, but a proof that, as @whuber says, the effort is doomed without more information.

The correlation is defined as:

\begin{equation} corr(X,Y) = \frac{cov(X,Y)}{\sqrt{var(X)var(Y)}} \quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad (1) \end{equation}

$var(Y)$ is known, so it's only necessary to estimate $var(X)$ and $cov(X,Y)$.

Let $Z$ index sub-populations. The following equalities are proved in introductory stat theory classes.

\begin{equation} var(X) = E_Z(var(X|Z)) + var_Z(E(X|Z) \quad\quad\quad\quad\quad\quad\quad\quad (2) \end{equation}

\begin{equation} cov(X,Y) = E_Z(cov(X,Y)|Z) + cov_Z(E(X|Z), E(Y|Z)) \quad\quad\quad (3) \end{equation}

Here $E_Z$ means that expectation is taken over the distribution of sub-populations, and a similar definition applies to $var_Z$ and $cov_Z$. The $|$ indicates a within-sub-population distribution; for example, $E(X|Z)$ is the mean of X in sub=population Z.

The second term in Eqn 2 and in Eqn 3 depends on the sub-population means of $X$; so both terms can be estimated. However the first term in each equation depends on the within-sub-population distribution of X (Eqn 2) and (X,Y) (Eqn 3), so neither term can be estimated from the available data.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.