# Correlation between realizations of normal-normal random variable

New on the site and could really use some help.

I'm stuck on a problem that should be obvious.

We have two random variables, $\Theta$ and $X$, where the mean of one is the realization of the other:

$\Theta \sim N\left (\mu ,\frac{\sigma^2}{m} \right )$

$X \sim N\left (\Theta ,\frac{\sigma^2}{n} \right )$

We are looking for the correlation between two realizations of $X$, $x_i$ and $x_j$ for a given value of m.

The solutions should be:

$corr(x_i,x_j) = \frac{n}{m+n}$

but I must be making a dumb mistake as I am unable to obtain this result. Could somebody please show how to get this simple result?

• You need to add the self-study tag. May 18, 2017 at 17:17
• What have you done so far? May 18, 2017 at 17:21
• I came across it in a paper I am reading, but have not done any maths related to probability theory in a long time. Managed to get all their other results, am just stuck on the correlation. Assumed it would need to be done with a formula of the sort: $\rho = \frac{cov(X,Y)}{var(X)var(Y)}$ Given that they are from the same distribution I assumed that this formula could be simplified, but then I just get $\rho = 1/(var(X))$ which does not work. Played around with the problem a lot more after that, but nothing I do gives the result.
– DC12
May 18, 2017 at 17:40
• $var(X) = \sigma^2/n$ and Cov(X,Y) does not equal 1. May 18, 2017 at 17:44
• Ok. It seems the result comes from $\rho=\frac{cov(x_i,x_j)}{\sqrt(var(X)^2)}$, if we set $cov(x_i,x_j) = \frac{\sigma^2}{n+m}$. Is this the correct approach of getting the result? Why is the covariance equal to $\frac{\sigma^2}{n+m}$.
– DC12
May 18, 2017 at 18:41

When you have this hierarchical structure, you usually want the law of total expectation/variance/covariance. This also looks like a factor model. You might want to add that tag.

Clearly, the variance of X given $m$ is $\text{Var}(X) = \frac{\sigma^2}{n}$

No, $\text{Var}(X) = \text{Var}[E(X|\Theta)] + E[\text{Var}(X|\Theta)] = \frac{\sigma^2}{m} +\frac{\sigma^2}{n}$ by the law of total variance.

If we integrate the two distributions so that we get $X \sim N\left (\mu,\frac{\sigma^2}{n+m} \right )$

Your answer here contradicts your answer above. Yes, normal, yes mean $\mu$, but different variance. See above.

The covariance between realizations xi and xj of X are given by: $cov(x_i,x_j)=E[X^2]-E[X]^2 = \frac{\sigma^2}{m+n}+\mu^2-\mu^2=\frac{\sigma^2}{m+n}$

I'm assuming your $X$ realizations are conditionally independent, given $\Theta$ here. But you want to use the law of total covariance. $\text{Cov}(X_i,X_j) = E[\text{Cov}(X_i,X_j|\Theta)] + \text{Cov}[E(X_i|\Theta),E(X_i|\theta)] = \text{Var}(\theta) = \sigma^2/m$. You can take it from here.

• Thanks. I was just simulating it in R and seeing that I was tremendously off in the previous post. You are absolutely right.
– DC12
May 18, 2017 at 23:08

THE SOLUTION IN THIS POST IS MISTAKEN. I DO NOT DELETE IT AS TAYLOR'S RESPONSE TO THIS ANSWER POST GIVES THE CORRECT ANSWER AND MAKES MORE SENSE IF MY MISTAKEN ANSWER REMAINS AVAILABLE.

OK... I think it is largely solved with Michael's help above.

Clearly, the variance of X given m is $var(X) = \frac{\sigma^2}{n}$.

If we integrate the two distributions so that we get:

$X \sim N\left (\mu,\frac{\sigma^2}{n+m} \right )$

The covariance between realizations $x_i$ and $x_j$ of X are given by:

$cov(x_i,x_j)=E[X^2]-E[X]^2 = \frac{\sigma^2}{m+n}+\mu^2-\mu^2=\frac{\sigma^2}{m+n}$

Plotting this in the standard correlation formula, we get:

$corr(x_i,x_j) = \frac{cov(x_i,x_j)}{\sqrt(var(x_i)+var(x_j))} = \frac{n}{m+n}$

Which is the result we are looking for... the only thing I do not entirely get is why we use $var(X)$, the variance of N conditional on $\Theta$ to compute the variance, but compute the covariance independent of $\Theta$. If somebody could explain this that would be great!

• And if somebody confirm whether the above is correct that could also be great.
– DC12
May 18, 2017 at 19:15