1
$\begingroup$

New on the site and could really use some help.

I'm stuck on a problem that should be obvious.

We have two random variables, $\Theta$ and $X$, where the mean of one is the realization of the other:

$\Theta \sim N\left (\mu ,\frac{\sigma^2}{m} \right )$

$X \sim N\left (\Theta ,\frac{\sigma^2}{n} \right )$

We are looking for the correlation between two realizations of $X$, $x_i$ and $x_j$ for a given value of m.

The solutions should be:

$corr(x_i,x_j) = \frac{n}{m+n} $

but I must be making a dumb mistake as I am unable to obtain this result. Could somebody please show how to get this simple result?

$\endgroup$
5
  • $\begingroup$ You need to add the self-study tag. $\endgroup$ May 18, 2017 at 17:17
  • $\begingroup$ What have you done so far? $\endgroup$ May 18, 2017 at 17:21
  • $\begingroup$ I came across it in a paper I am reading, but have not done any maths related to probability theory in a long time. Managed to get all their other results, am just stuck on the correlation. Assumed it would need to be done with a formula of the sort: $\rho = \frac{cov(X,Y)}{var(X)var(Y)}$ Given that they are from the same distribution I assumed that this formula could be simplified, but then I just get $\rho = 1/(var(X))$ which does not work. Played around with the problem a lot more after that, but nothing I do gives the result. $\endgroup$
    – DC12
    May 18, 2017 at 17:40
  • $\begingroup$ $var(X) = \sigma^2/n$ and Cov(X,Y) does not equal 1. $\endgroup$ May 18, 2017 at 17:44
  • $\begingroup$ Ok. It seems the result comes from $\rho=\frac{cov(x_i,x_j)}{\sqrt(var(X)^2)}$, if we set $cov(x_i,x_j) = \frac{\sigma^2}{n+m}$. Is this the correct approach of getting the result? Why is the covariance equal to $\frac{\sigma^2}{n+m}$. $\endgroup$
    – DC12
    May 18, 2017 at 18:41

2 Answers 2

1
$\begingroup$

When you have this hierarchical structure, you usually want the law of total expectation/variance/covariance. This also looks like a factor model. You might want to add that tag.

Clearly, the variance of X given $m$ is $\text{Var}(X) = \frac{\sigma^2}{n}$

No, $\text{Var}(X) = \text{Var}[E(X|\Theta)] + E[\text{Var}(X|\Theta)] = \frac{\sigma^2}{m} +\frac{\sigma^2}{n}$ by the law of total variance.

If we integrate the two distributions so that we get $X \sim N\left (\mu,\frac{\sigma^2}{n+m} \right )$

Your answer here contradicts your answer above. Yes, normal, yes mean $\mu$, but different variance. See above.

The covariance between realizations xi and xj of X are given by: $cov(x_i,x_j)=E[X^2]-E[X]^2 = \frac{\sigma^2}{m+n}+\mu^2-\mu^2=\frac{\sigma^2}{m+n}$

I'm assuming your $X$ realizations are conditionally independent, given $\Theta$ here. But you want to use the law of total covariance. $\text{Cov}(X_i,X_j) = E[\text{Cov}(X_i,X_j|\Theta)] + \text{Cov}[E(X_i|\Theta),E(X_i|\theta)] = \text{Var}(\theta) = \sigma^2/m$. You can take it from here.

$\endgroup$
1
  • $\begingroup$ Thanks. I was just simulating it in R and seeing that I was tremendously off in the previous post. You are absolutely right. $\endgroup$
    – DC12
    May 18, 2017 at 23:08
1
$\begingroup$

THE SOLUTION IN THIS POST IS MISTAKEN. I DO NOT DELETE IT AS TAYLOR'S RESPONSE TO THIS ANSWER POST GIVES THE CORRECT ANSWER AND MAKES MORE SENSE IF MY MISTAKEN ANSWER REMAINS AVAILABLE.

OK... I think it is largely solved with Michael's help above.

Clearly, the variance of X given m is $var(X) = \frac{\sigma^2}{n}$.

If we integrate the two distributions so that we get:

$X \sim N\left (\mu,\frac{\sigma^2}{n+m} \right )$

The covariance between realizations $x_i$ and $x_j$ of X are given by:

$cov(x_i,x_j)=E[X^2]-E[X]^2 = \frac{\sigma^2}{m+n}+\mu^2-\mu^2=\frac{\sigma^2}{m+n}$

Plotting this in the standard correlation formula, we get:

$corr(x_i,x_j) = \frac{cov(x_i,x_j)}{\sqrt(var(x_i)+var(x_j))} = \frac{n}{m+n} $

Which is the result we are looking for... the only thing I do not entirely get is why we use $var(X)$, the variance of N conditional on $\Theta$ to compute the variance, but compute the covariance independent of $\Theta$. If somebody could explain this that would be great!

$\endgroup$
1
  • $\begingroup$ And if somebody confirm whether the above is correct that could also be great. $\endgroup$
    – DC12
    May 18, 2017 at 19:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.