Correlation between realizations of normal-normal random variable New on the site and could really use some help.
I'm stuck on a problem that should be obvious.
We have two random variables, $\Theta$ and $X$, where the mean of one is the realization of the other:
$\Theta \sim N\left (\mu ,\frac{\sigma^2}{m}  \right )$
$X \sim N\left (\Theta  ,\frac{\sigma^2}{n}  \right )$
We are looking for the correlation between two realizations of $X$, $x_i$ and $x_j$ for a given value of m.
The solutions should be:
$corr(x_i,x_j) = \frac{n}{m+n} $
but I must be making a dumb mistake as I am unable to obtain this result. Could somebody please show how to get this simple result?
 A: THE SOLUTION IN THIS POST IS MISTAKEN. I DO NOT DELETE IT AS TAYLOR'S RESPONSE TO THIS ANSWER POST GIVES THE CORRECT ANSWER AND MAKES MORE SENSE IF MY MISTAKEN ANSWER REMAINS AVAILABLE.
OK... I think it is largely solved with Michael's help above.
Clearly, the variance of X given m is $var(X) = \frac{\sigma^2}{n}$.
If we integrate the two distributions so that we get:
$X \sim N\left (\mu,\frac{\sigma^2}{n+m}  \right )$
The covariance between realizations $x_i$ and $x_j$ of X are given by:
$cov(x_i,x_j)=E[X^2]-E[X]^2 = \frac{\sigma^2}{m+n}+\mu^2-\mu^2=\frac{\sigma^2}{m+n}$
Plotting this in the standard correlation formula, we get:
$corr(x_i,x_j) = \frac{cov(x_i,x_j)}{\sqrt(var(x_i)+var(x_j))} = \frac{n}{m+n} $
Which is the result we are looking for... the only thing I do not entirely get is why we use $var(X)$, the variance of N conditional on $\Theta$ to compute the variance, but compute the covariance independent of $\Theta$. If somebody could explain this that would be great!
A: When you have this hierarchical structure, you usually want the law of total expectation/variance/covariance. This also looks like a factor model. You might want to add that tag.

Clearly, the variance of X given $m$ is $\text{Var}(X) = \frac{\sigma^2}{n}$

No, $\text{Var}(X) = \text{Var}[E(X|\Theta)] + E[\text{Var}(X|\Theta)] = \frac{\sigma^2}{m} +\frac{\sigma^2}{n}$ by the law of total variance.

If we integrate the two distributions so that we get $X \sim N\left (\mu,\frac{\sigma^2}{n+m}  \right )$

Your answer here contradicts your answer above. Yes, normal, yes mean $\mu$, but different variance. See above.

The covariance between realizations xi and xj of X are given by:
  $cov(x_i,x_j)=E[X^2]-E[X]^2 =
 \frac{\sigma^2}{m+n}+\mu^2-\mu^2=\frac{\sigma^2}{m+n}$

I'm assuming your $X$ realizations are conditionally independent, given $\Theta$ here. But you want to use the law of total covariance. $\text{Cov}(X_i,X_j) = E[\text{Cov}(X_i,X_j|\Theta)] + \text{Cov}[E(X_i|\Theta),E(X_i|\theta)] = \text{Var}(\theta) = \sigma^2/m$. You can take it from here.
