# Basic Bayesian reasoning question — what is wrong with this “equational” argument?

Let's say that we have a model:

$y + x = \epsilon$, $\epsilon \sim N(0, 1)$.

After observing a value for $y$, we can write $x$ as:

$x = \epsilon - y$

Since $y$ is just a constant, and $\epsilon$ is standard normally distributed, it seems that we can claim:

$\epsilon - y \sim N(-y, 1)$

and thus:

$p(x \mid y) = N(x; -y, 1)$ where $N$ is the normal pdf.

But this makes no sense as there is no mention of a prior for $x$ anywhere. So what went wrong with this "equational" reasoning argument?

-
Note: You haven't said anything about whether the random variables X and Y are i.i.d or not... – Steve S Aug 12 '14 at 1:26

There are two ways to interpret your statement $\epsilon \sim \mathcal N(0, 1)$. If $\epsilon \sim \mathcal N(0, 1)$ is taken to mean that $\epsilon$ is marginally $\mathcal N(0, 1)$ then your logic falls apart at concluding that $[\epsilon - y \mid Y = y] \sim \mathcal N(-y, 1)$. This is because there is no reason whatsover to conclude that $[\epsilon \mid Y = y] \sim \mathcal N(0, 1)$; the assumption is made about the marginal distribution of $\epsilon$, but if $\epsilon$ and $Y$ are dependent we cannot make such a strong conclusion about the conditional distribution of $[\epsilon \mid Y]$.

Now, on the other hand, if one interprets $\epsilon \sim \mathcal N(0, 1)$ to mean that $[\epsilon \mid Y = y] \sim \mathcal N(0, 1)$ then your logic works. This is a standard assumption in parametric linear regression, after relabeling the variables slightly; one assumes $Y = X\beta + \epsilon$ and assumes that $[\epsilon \mid X] \sim \mathcal N(0, \sigma)$, from which is follows that $[Y \mid X] \sim \mathcal N(X\beta, \sigma)$.

-

"Since y is just a constant..."..it is not a constant. It is a sample from a distribution. The errors follow!!

-
If you're computing the conditional distribution of $x$ given the data, $f(x|y)$, as would often be done in a Bayesian context, $y$ is constant. – Glen_b Aug 12 '14 at 1:44
The equation mentioned is for x and not x|y. There is no mention of a marginal distribution – Sid Aug 12 '14 at 2:13
There's a mention of $p(x \mid y)$ right there in the question. – Glen_b Aug 12 '14 at 2:18
You're right...my bad!! serves me right for looking up problems during the hangries!! – Sid Aug 12 '14 at 4:42
I guess my main problem was with putting up $x=\epsilon - y$ followed by "since y is just a constant..." I read that and didn't proceed further – Sid Aug 12 '14 at 4:45

I don't see any problem in your reasoning, except you want a prior on $x$. Note that $p(x,y,\epsilon) = p(x|y,\epsilon)p(y)p(\epsilon)$.

-
Hmmm, so what would be the correct equation for $p(x \mid y)$ that has a $p(x)$ or $p(y)$ term in it? $p(x, y, \epsilon) = 1$ iff $x + y = \epsilon$, right? I'm not seeing how it's helpful? – yong Aug 12 '14 at 1:37