# Reversing conditional distribution

I want to find $p(x \vert y)$ given $p(y \vert x)$. I am aware of Bayesian formula $p(x \vert y) = \frac{p(y \vert x)p(x)}{\int_{-\infty}^{+\infty}p(y \vert x)p(x)dx}$ but I can not understand why the following logic is wrong.

Let $x$ be some random variable with pdf $p(x)$ and $\varepsilon \sim N(0,1)$ independent of $x$.

Define $$y(x) \equiv x + \varepsilon\tag{1},$$ then $(y\vert x) \sim N(x,1)$.

But it also should be true that $$x=y-\varepsilon\tag{2}$$ and because standard normal distribution is symmetric around zero, then it should follow that $(x \vert y) \sim N(y,1)$. Thus we didn't use the information about unconditional distribution $p(x)$, needed for Bayesian formula!

Please, point me where I am wrong.

The following analytical solution should prove that two distributions are indeed not equal.

Let $x \sim N(\mu,\sigma)$, then $p(x)=\frac{e^{-\frac{(x-\mu )^2}{2 \sigma ^2}}}{\sqrt{2 \pi } \sigma }$.

Let $y(x)=x+\varepsilon$ and $\varepsilon \sim N(0,1)$, then $p(y\vert x)=\frac{e^{-\frac{1}{2} (y-x)^2}}{\sqrt{2 \pi }}$. Hence, $\int_{-\infty}^{+\infty}p(y \vert x)p(x)dx=\frac{e^{-\frac{(y-\mu )^2}{2 \left(\sigma ^2+1\right)}}}{\sqrt{2 \pi } \sqrt{\frac{1}{\sigma ^2}+1} \sigma }$ and therefore by Bayesian formula $p(x\vert y)=\frac{\sqrt{\frac{1}{\sigma ^2}+1} \exp \left(-\frac{(x-\mu )^2}{2 \sigma ^2}-\frac{1}{2} (y-x)^2+\frac{(y-\mu )^2}{2 \left(\sigma ^2+1\right)}\right)}{\sqrt{2 \pi }}$. Finally, this is not equal to pdf of $N(y,1)$, which is $\frac{e^{-\frac{1}{2} (x-y)^2}}{\sqrt{2 \pi }}$.

• This is the fiducial fallacy! – Xi'an Jan 10 '18 at 15:29

Your equation (1) implies that $y$ and $\varepsilon$ are dependent random variables. Hence, the distribution of $\varepsilon$ in equation (2) depends on $y$ and is no longer N(0,1) conditional on $y$ and $x=y-\varepsilon$ is not $N(y,1)$.