Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Let's say that we have a model:

$y + x = \epsilon$, $\epsilon \sim N(0, 1)$.

After observing a value for $y$, we can write $x$ as:

$x = \epsilon - y$

Since $y$ is just a constant, and $\epsilon$ is standard normally distributed, it seems that we can claim:

$\epsilon - y \sim N(-y, 1)$

and thus:

$p(x \mid y) = N(x; -y, 1)$ where $N$ is the normal pdf.

But this makes no sense as there is no mention of a prior for $x$ anywhere. So what went wrong with this "equational" reasoning argument?

share|improve this question
Note: You haven't said anything about whether the random variables X and Y are i.i.d or not... – Steve S Aug 12 '14 at 1:26
up vote 4 down vote accepted

There are two ways to interpret your statement $\epsilon \sim \mathcal N(0, 1)$. If $\epsilon \sim \mathcal N(0, 1)$ is taken to mean that $\epsilon$ is marginally $\mathcal N(0, 1)$ then your logic falls apart at concluding that $[\epsilon - y \mid Y = y] \sim \mathcal N(-y, 1)$. This is because there is no reason whatsover to conclude that $[\epsilon \mid Y = y] \sim \mathcal N(0, 1)$; the assumption is made about the marginal distribution of $\epsilon$, but if $\epsilon$ and $Y$ are dependent we cannot make such a strong conclusion about the conditional distribution of $[\epsilon \mid Y]$.

Now, on the other hand, if one interprets $\epsilon \sim \mathcal N(0, 1)$ to mean that $[\epsilon \mid Y = y] \sim \mathcal N(0, 1)$ then your logic works. This is a standard assumption in parametric linear regression, after relabeling the variables slightly; one assumes $Y = X\beta + \epsilon$ and assumes that $[\epsilon \mid X] \sim \mathcal N(0, \sigma)$, from which is follows that $[Y \mid X] \sim \mathcal N(X\beta, \sigma)$.

share|improve this answer

"Since y is just a constant..." is not a constant. It is a sample from a distribution. The errors follow!!

share|improve this answer
If you're computing the conditional distribution of $x$ given the data, $f(x|y)$, as would often be done in a Bayesian context, $y$ is constant. – Glen_b Aug 12 '14 at 1:44
The equation mentioned is for x and not x|y. There is no mention of a marginal distribution – Sid Aug 12 '14 at 2:13
There's a mention of $p(x \mid y) $ right there in the question. – Glen_b Aug 12 '14 at 2:18
You're bad!! serves me right for looking up problems during the hangries!! – Sid Aug 12 '14 at 4:42
I guess my main problem was with putting up $x=\epsilon - y$ followed by "since y is just a constant..." I read that and didn't proceed further – Sid Aug 12 '14 at 4:45

I don't see any problem in your reasoning, except you want a prior on $x$. Note that $p(x,y,\epsilon) = p(x|y,\epsilon)p(y)p(\epsilon)$.

share|improve this answer
Hmmm, so what would be the correct equation for $p(x \mid y)$ that has a $p(x)$ or $p(y)$ term in it? $p(x, y, \epsilon) = 1$ iff $x + y = \epsilon$, right? I'm not seeing how it's helpful? – yong Aug 12 '14 at 1:37

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.