Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Let's say that we have a model:

$y + x = \epsilon$, $\epsilon \sim N(0, 1)$.

After observing a value for $y$, we can write $x$ as:

$x = \epsilon - y$

Since $y$ is just a constant, and $\epsilon$ is standard normally distributed, it seems that we can claim:

$\epsilon - y \sim N(-y, 1)$

and thus:

$p(x \mid y) = N(x; -y, 1)$ where $N$ is the normal pdf.

But this makes no sense as there is no mention of a prior for $x$ anywhere. So what went wrong with this "equational" reasoning argument?

share|improve this question
    
Note: You haven't said anything about whether the random variables X and Y are i.i.d or not... –  Steve S Aug 12 at 1:26

3 Answers 3

up vote 4 down vote accepted

There are two ways to interpret your statement $\epsilon \sim \mathcal N(0, 1)$. If $\epsilon \sim \mathcal N(0, 1)$ is taken to mean that $\epsilon$ is marginally $\mathcal N(0, 1)$ then your logic falls apart at concluding that $[\epsilon - y \mid Y = y] \sim \mathcal N(-y, 1)$. This is because there is no reason whatsover to conclude that $[\epsilon \mid Y = y] \sim \mathcal N(0, 1)$; the assumption is made about the marginal distribution of $\epsilon$, but if $\epsilon$ and $Y$ are dependent we cannot make such a strong conclusion about the conditional distribution of $[\epsilon \mid Y]$.

Now, on the other hand, if one interprets $\epsilon \sim \mathcal N(0, 1)$ to mean that $[\epsilon \mid Y = y] \sim \mathcal N(0, 1)$ then your logic works. This is a standard assumption in parametric linear regression, after relabeling the variables slightly; one assumes $Y = X\beta + \epsilon$ and assumes that $[\epsilon \mid X] \sim \mathcal N(0, \sigma)$, from which is follows that $[Y \mid X] \sim \mathcal N(X\beta, \sigma)$.

share|improve this answer

"Since y is just a constant..."..it is not a constant. It is a sample from a distribution. The errors follow!!

share|improve this answer
2  
If you're computing the conditional distribution of $x$ given the data, $f(x|y)$, as would often be done in a Bayesian context, $y$ is constant. –  Glen_b Aug 12 at 1:44
    
The equation mentioned is for x and not x|y. There is no mention of a marginal distribution –  Sid Aug 12 at 2:13
1  
There's a mention of $p(x \mid y) $ right there in the question. –  Glen_b Aug 12 at 2:18
    
You're right...my bad!! serves me right for looking up problems during the hangries!! –  Sid Aug 12 at 4:42
    
I guess my main problem was with putting up $x=\epsilon - y$ followed by "since y is just a constant..." I read that and didn't proceed further –  Sid Aug 12 at 4:45

I don't see any problem in your reasoning, except you want a prior on $x$. Note that $p(x,y,\epsilon) = p(x|y,\epsilon)p(y)p(\epsilon)$.

share|improve this answer
    
Hmmm, so what would be the correct equation for $p(x \mid y)$ that has a $p(x)$ or $p(y)$ term in it? $p(x, y, \epsilon) = 1$ iff $x + y = \epsilon$, right? I'm not seeing how it's helpful? –  yong Aug 12 at 1:37

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.