Finding the conditional distribution from given normal distributions using Bayes' theorem

Question

Background

This question is related to my previous question: Describing the measurement of a random variable as another random variable, but I've narrowed and clarified my question. I think I've modified my question enough to warrant this new question.

Suppose we have a box of resistors. The manufacturer rates these resistors at 100 ohms, but they have some variability. Let $x$ be the true resistance of a resistor chosen from the box at random. Suppose $x$ is normally distributed

$$ x \sim N(\mu_x,\sigma_x^2) $$

We have an ohmmeter that can measure the resistance of the selected resistor. Denote the measurement of the resistor as $y=x+\epsilon$, where the error term $\epsilon$ is normally distributed with mean zero
$$\epsilon \sim N(0,\sigma_\epsilon^2)$$

from these given distributions, we can reason

\begin{align*} y &\sim N(\mu_y,\sigma_y^2) = N(\mu_x,\sigma_x^2 + \sigma_\epsilon^2)\\ y|x &\sim N(x,\sigma_\epsilon^2) \end{align*}

Question

My question is how to find the conditional distribution $x|y$. I think I can use Bayes' theorem

$$f(x,y) = f(x|y)f(y) = f(y|x)f(x)$$

which yields

\begin{align*} f(x|y)&=f(y|x)f(x)/f(y)\\ &= \frac{1}{\sigma_\epsilon\sqrt{2\pi}}e^{-\frac{1}{2}\left(\frac{y-x}{\sigma_\epsilon}\right)^2} \frac{1}{\sigma_x\sqrt{2\pi}}e^{-\frac{1}{2}\left(\frac{x-\mu_x}{\sigma_x}\right)^2} / \left(\frac{1}{\sqrt{\sigma_x^2+\sigma_\epsilon^2}\sqrt{2\pi}}\right)e^{-\frac{1}{2}\left(\frac{y-\mu_y}{\sqrt{\sigma_x^2+\sigma_\epsilon^2}}\right)^2}\\ &=\frac{\sqrt{\sigma_x^2+\sigma_\epsilon^2}}{\sigma_x\sigma_\epsilon\sqrt{2\pi}}e^{-\frac{1}{2}\left(\left(\frac{y-x}{\sigma_\epsilon}\right)^2+\left(\frac{x-\mu_x}{\sigma_x}\right)^2-\left(\frac{y-\mu_y}{\sqrt{\sigma_x^2+\sigma_\epsilon^2}}\right)^2\right)} \end{align*}

Is this a normal distribution? I tried expanding the quadratics inside the exponential and combining the fractions with common denominator $\sigma_x^2\sigma_\epsilon^2(\sigma_x^2+\sigma_\epsilon^2)$. Below, I've written how the numerator expands (utilizing $\mu_y=\mu_x$) and cancelling some terms:

\begin{align*} (y-2xy+x^2)(\sigma_x^4+\sigma_x^2\sigma_\epsilon^2)+(x-2\mu_x x + \mu_x^2)(\sigma_\epsilon^2\sigma_x^2+\sigma_\epsilon^4)-(y^2-2\mu_y y + \mu_y^2)(\sigma_x^2\sigma_\epsilon^2)\\ = y^2\sigma_x^4-2xy\sigma_x^4-2xy\sigma_\epsilon^2\sigma_x^2+x^2\sigma_x^4 + x^2\sigma_\epsilon^2\sigma_x^2+x^2\sigma_\epsilon^4+x^2\sigma_\epsilon^2\sigma_x^2-2\mu_x x \sigma_\epsilon^4-2\mu_x x \sigma_\epsilon^2\sigma_x^2 + \mu_x \sigma_\epsilon^2 +2\mu_x y \sigma_x^2\sigma_\epsilon^2 \end{align*}

I'm was hoping for more cancellation, so I'm not sure how I could factor this.

Is there a better way to find the conditional distribution $x|y$, and maybe to verify that it is a normal distribution?

Answer 1

Thanks to @j-delaney for the hint on completing the square.

We have

\begin{align*} f(x|y)&=\frac{\sqrt{\sigma_x^2+\sigma_\epsilon^2}}{\sigma_x\sigma_\epsilon\sqrt{2\pi}}e^{-\frac{1}{2}\left(\left(\frac{y-x}{\sigma_\epsilon}\right)^2+\left(\frac{x-\mu_x}{\sigma_x}\right)^2-\left(\frac{y-\mu_y}{\sqrt{\sigma_x^2+\sigma_\epsilon^2}}\right)^2\right)}\\ \end{align*}

Consider just the argument of the exponential, and ignore the factor of $-1/2$ for now.

\begin{align*} A&=\left(\frac{y-x}{\sigma_\epsilon}\right)^2+\left(\frac{x-\mu_x}{\sigma_x}\right)^2-\left(\frac{y-\mu_y}{\sqrt{\sigma_x^2+\sigma_\epsilon^2}}\right)^2\\ &=\frac{(\sigma_x^2+\sigma_\epsilon^2)x^2-2(\sigma_x^2y+\sigma_\epsilon^2\mu_x)x+\sigma_x^2y^2+\sigma_\epsilon^2\mu_x^2}{\sigma_x^2\sigma_\epsilon^2}-\left(\frac{y-\mu_y}{\sqrt{\sigma_x^2+\sigma_\epsilon^2}}\right)^2\\ &=\text{complete the square and simplify}...\\ &=\frac{\left(\sqrt{\sigma_x^2+\sigma_\epsilon^2}x-\frac{\sigma_x^2y+\sigma_\epsilon^2\mu_x}{\sqrt{\sigma_x^2+\sigma_\epsilon^2}}\right)^2}{\sigma_x^2\sigma_\epsilon^2}+\left(\frac{y-\mu_y}{\sqrt{\sigma_x^2+\sigma_\epsilon^2}}\right)^2-\left(\frac{y-\mu_y}{\sqrt{\sigma_x^2+\sigma_\epsilon^2}}\right)^2\\ &=\left(\frac{x-\frac{\sigma_x^2y+\sigma_\epsilon^2\mu_x}{\sigma_x^2+\sigma_\epsilon^2}}{\frac{\sigma_x\sigma_\epsilon}{\sqrt{\sigma_x^2+\sigma_\epsilon^2}}}\right)^2 \end{align*}

So the entire PDF of $x|y$ can be written

$$ f(x|y)=\frac{1}{\frac{\sigma_x\sigma_\epsilon}{\sqrt{\sigma_x^2+\sigma_\epsilon^2}}\sqrt{2\pi}}e^{-\frac{1}{2}\left(\frac{x-\frac{\sigma_x^2y+\sigma_\epsilon^2\mu_x}{\sigma_x^2+\sigma_\epsilon^2}}{\frac{\sigma_x\sigma_\epsilon}{\sqrt{\sigma_x^2+\sigma_\epsilon^2}}}\right)^2} $$

which implies $$x|y \sim N(\mu_{x|y},\sigma_{x|y}^2)$$ with \begin{align*} \mu_{x|y}&=\frac{\sigma_x^2y+\sigma_\epsilon^2\mu_x}{\sigma_x^2+\sigma_\epsilon^2}\\ \sigma_{x|y}&=\frac{\sigma_x\sigma_\epsilon}{\sqrt{\sigma_x^2+\sigma_\epsilon^2}} \end{align*}

The implication of this distribution could be interpreted as the answer to the question "How should I estimate the true value of $x$ given the measurement $y$? The most likely value or best estimation of $x$ given $y$ is the expected value $E(x|y)=\mu_{x|y}$.