2
$\begingroup$

Say I know the conditional distribution:

$Y|Z \sim\mathbb{N}(Z,\sigma^2)$

Now, what if I reversed this though and wanted to find the conditional distribution $Z|Y$?

From intuition I would expect the distribution to still be normal with mean of $Y$ (i.e., $Y$ is an unbiased predictor of $Z$?), but I would also think the variance is larger.

$\endgroup$
6
  • 7
    $\begingroup$ You can't really say anything about the distribution of $Z|Y$ without making some assumptions on $P(Z)$ and $P(Y)$. $Z$ could be a constant, or a die roll, or Poisson or anything you can imagine. And once you state something about $P(Z)$ and $P(Y)$ then Bayes theorem will give the answer. $\endgroup$
    – Corvus
    Commented Feb 19, 2016 at 17:24
  • $\begingroup$ This is the answer to the question, you might as well paste the same thing as an answer $\endgroup$
    – Bach
    Commented Feb 19, 2016 at 17:30
  • $\begingroup$ @Corone Hmm, so are you saying I would need to know something like Z ~ $\Bbb N$($\mu$,$\sigma_z^2$) to be able to fully specify the answer? $\endgroup$
    – JPJ
    Commented Feb 19, 2016 at 18:39
  • 1
    $\begingroup$ @user3496060 More precisely, you need to know both the marginal distributions of Z and of Y. $\endgroup$
    – Sycorax
    Commented Feb 19, 2016 at 18:42
  • $\begingroup$ OK, so like $f_y$(Y|Z) = $f_z$(Z|Y)$f_y(Y)\frac1{f_z(Z)}$ $\endgroup$
    – JPJ
    Commented Feb 19, 2016 at 18:52

1 Answer 1

2
$\begingroup$

You can't really say anything about the distribution of $Z|Y$ without making some assumptions on $P(Z)$ and $P(Y)$. $Z$ could be a constant, or a die roll, or Poisson or anything you can imagine. And once you state something about $P(Z)$ and $P(Y)$ then Bayes theorem will give the answer:

$P(Y|Z) = \frac{P(Z|Y)P(Y)}{P(Z)}$

In the special case where $Y$ and $Z$ are both normally distribution, then Y|Z being normal with mean proportional to $Z$ fixed variance is sufficient to conclude that $Y$ and $Z$ are jointly normal. In this case:

$Z|Y \sim N(\mu_Z + \frac{\sigma_Z \rho}{\sigma_Y} (Y-\mu_Y),(1-\rho^2)\sigma_z^2 )$

Notice that the distribution is NOT centred around $Y$ as you supposed - this means that if the Y you observe is very high above the mean of Z, then it is likely that that was due to a quite high value of Y combined with a quite high value of Z, rather than an extremely high value of Z. This effect is the source of regression toward the mean

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.