Say we have a multivariate normal distribution with non-zero covariance $\alpha$

$X \sim N \left(\mu = \left({\begin{array}{cc} 0 \\ 1 \end{array} } \right), \Sigma = \left({\begin{array}{cc} 1 & \alpha \\ \alpha & 1 \\ \end{array} } \right) \right)$

Now we take $m$ samples of same size $n$. Obviously the expected correlation of the two elements of $X$ in each sample of size $n$ is $\rho_1 = \rho_2 = ... = \rho_m = \alpha/1^2 = \alpha$.

Now suppose we create new random variables by taking the average $\bar{X} = \left({\begin{array}{cc} \bar{x_1} \\ \bar{x_2} \end{array} } \right)$ in each sample.

What is the expected correlation of the two elements of $\bar{X}$ among the $m$ data points we have now?

My intuitive response was that it is $\alpha$ as well, but after some thinking and simulations, I don't trust that intuition anymore. Approaching it formally, my idea would be to derive the formula for their covariance as

$\text{Cov}(\bar{x_1}, \bar{x_2}) = E(\bar{x_1} \bar{x_2}) - E(\bar{x_1}) \ E(\bar{x_2}) $

Once I know this covariance, I can calculate the correlation, since I know the variances of both $\bar{x}_1$ and $\bar{x}_2$ by the CLT, specifically here $\text{Var}(\bar{x}_1) = \text{Var} (\bar{x}_2) = 1/n$. However, I don't see any way to get to $E(\bar{x_1} \bar{x_2})$, which I need to calculate the covariance, which does not involve the covariance itself. For example, calculating the integral mentioned here requires a full specification of the joint PDF including covariance.

I am probably already off track though.


1 Answer 1


As in the multivariate setting, the variance-covariance matrix of a sample mean vector is:

$$ \operatorname{var}(\bar{\bf{x}})=\frac{1}{n}\Sigma $$ where $\Sigma$ is the population variance-covariance matrix. Hence, the distribution of the mean vector is multivariate normal: $$ \bar{\bf{x}}\sim \operatorname{N}\left(\bf{\mu}, \frac{1}{n}\Sigma\right) $$ where $\bf{\mu}$ is the population mean vector. In your case, $\Sigma = \left({\begin{array}{cc} 1 & \alpha \\ \alpha & 1 \\ \end{array} } \right) $ and $\frac{1}{n}\Sigma = \left({\begin{array}{cc} 1/n & \alpha/n \\ \alpha/n & 1/n \\ \end{array} } \right)$. So for the correlation we have: $$ \operatorname{Corr}(\bar{x}_1, \bar{x}_2) = \frac{\operatorname{Cov}(\bar{x}_1, \bar{x}_2)}{\sqrt{\operatorname{Var}(\bar{x}_1)}\sqrt{\operatorname{Var}(\bar{x}_2)}} = \frac{\alpha/n}{(1/n)^{1/2}\cdot (1/n)^{1/2}} = \alpha $$

  • $\begingroup$ Excellent, thank you, I was missing the initial formula! I wonder if there can be any weird behavior when using the usual estimators with small n (50) and m (5) here? Thats what I did in my simulations (1000 reps) $\endgroup$
    – stefgehrig
    Jun 25, 2021 at 16:26
  • $\begingroup$ @stefgehrig The distribution of the correlation coefficient can be highly skewed for small $m$. For example, the mean of the distribution for $m=5$ and $\alpha = 0.5$ is about $0.452$, I think. $\endgroup$ Jun 25, 2021 at 17:44
  • $\begingroup$ Is m->inf required to hold for sample mean vector formula? $\endgroup$
    – stefgehrig
    Jun 26, 2021 at 9:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.