1
$\begingroup$

Let's say that we have a model:

$y + x = \epsilon$, $\epsilon \sim N(0, 1)$.

After observing a value for $y$, we can write $x$ as:

$x = \epsilon - y$

Since $y$ is just a constant, and $\epsilon$ is standard normally distributed, it seems that we can claim:

$\epsilon - y \sim N(-y, 1)$

and thus:

$p(x \mid y) = N(x; -y, 1)$ where $N$ is the normal pdf.

But this makes no sense as there is no mention of a prior for $x$ anywhere. So what went wrong with this "equational" reasoning argument?

$\endgroup$
1
  • $\begingroup$ Note: You haven't said anything about whether the random variables X and Y are i.i.d or not... $\endgroup$
    – Steve S
    Commented Aug 12, 2014 at 1:26

3 Answers 3

5
$\begingroup$

There are two ways to interpret your statement $\epsilon \sim \mathcal N(0, 1)$. If $\epsilon \sim \mathcal N(0, 1)$ is taken to mean that $\epsilon$ is marginally $\mathcal N(0, 1)$ then your logic falls apart at concluding that $[\epsilon - y \mid Y = y] \sim \mathcal N(-y, 1)$. This is because there is no reason whatsover to conclude that $[\epsilon \mid Y = y] \sim \mathcal N(0, 1)$; the assumption is made about the marginal distribution of $\epsilon$, but if $\epsilon$ and $Y$ are dependent we cannot make such a strong conclusion about the conditional distribution of $[\epsilon \mid Y]$.

Now, on the other hand, if one interprets $\epsilon \sim \mathcal N(0, 1)$ to mean that $[\epsilon \mid Y = y] \sim \mathcal N(0, 1)$ then your logic works. This is a standard assumption in parametric linear regression, after relabeling the variables slightly; one assumes $Y = X\beta + \epsilon$ and assumes that $[\epsilon \mid X] \sim \mathcal N(0, \sigma)$, from which is follows that $[Y \mid X] \sim \mathcal N(X\beta, \sigma)$.

$\endgroup$
0
$\begingroup$

"Since y is just a constant..."..it is not a constant. It is a sample from a distribution. The errors follow!!

$\endgroup$
5
  • 2
    $\begingroup$ If you're computing the conditional distribution of $x$ given the data, $f(x|y)$, as would often be done in a Bayesian context, $y$ is constant. $\endgroup$
    – Glen_b
    Commented Aug 12, 2014 at 1:44
  • $\begingroup$ The equation mentioned is for x and not x|y. There is no mention of a marginal distribution $\endgroup$
    – Sid
    Commented Aug 12, 2014 at 2:13
  • 1
    $\begingroup$ There's a mention of $p(x \mid y) $ right there in the question. $\endgroup$
    – Glen_b
    Commented Aug 12, 2014 at 2:18
  • $\begingroup$ You're right...my bad!! serves me right for looking up problems during the hangries!! $\endgroup$
    – Sid
    Commented Aug 12, 2014 at 4:42
  • $\begingroup$ I guess my main problem was with putting up $x=\epsilon - y$ followed by "since y is just a constant..." I read that and didn't proceed further $\endgroup$
    – Sid
    Commented Aug 12, 2014 at 4:45
0
$\begingroup$

I don't see any problem in your reasoning, except you want a prior on $x$. Note that $p(x,y,\epsilon) = p(x|y,\epsilon)p(y)p(\epsilon)$.

$\endgroup$
1
  • $\begingroup$ Hmmm, so what would be the correct equation for $p(x \mid y)$ that has a $p(x)$ or $p(y)$ term in it? $p(x, y, \epsilon) = 1$ iff $x + y = \epsilon$, right? I'm not seeing how it's helpful? $\endgroup$
    – yong
    Commented Aug 12, 2014 at 1:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.