0
$\begingroup$

Background

This question is related to my previous question: Describing the measurement of a random variable as another random variable, but I've narrowed and clarified my question. I think I've modified my question enough to warrant this new question.

Suppose we have a box of resistors. The manufacturer rates these resistors at 100 ohms, but they have some variability. Let $x$ be the true resistance of a resistor chosen from the box at random. Suppose $x$ is normally distributed

$$ x \sim N(\mu_x,\sigma_x^2) $$

We have an ohmmeter that can measure the resistance of the selected resistor. Denote the measurement of the resistor as $y=x+\epsilon$, where the error term $\epsilon$ is normally distributed with mean zero
$$\epsilon \sim N(0,\sigma_\epsilon^2)$$

from these given distributions, we can reason

\begin{align*} y &\sim N(\mu_y,\sigma_y^2) = N(\mu_x,\sigma_x^2 + \sigma_\epsilon^2)\\ y|x &\sim N(x,\sigma_\epsilon^2) \end{align*}

Question

My question is how to find the conditional distribution $x|y$. I think I can use Bayes' theorem

$$f(x,y) = f(x|y)f(y) = f(y|x)f(x)$$

which yields

\begin{align*} f(x|y)&=f(y|x)f(x)/f(y)\\ &= \frac{1}{\sigma_\epsilon\sqrt{2\pi}}e^{-\frac{1}{2}\left(\frac{y-x}{\sigma_\epsilon}\right)^2} \frac{1}{\sigma_x\sqrt{2\pi}}e^{-\frac{1}{2}\left(\frac{x-\mu_x}{\sigma_x}\right)^2} / \left(\frac{1}{\sqrt{\sigma_x^2+\sigma_\epsilon^2}\sqrt{2\pi}}\right)e^{-\frac{1}{2}\left(\frac{y-\mu_y}{\sqrt{\sigma_x^2+\sigma_\epsilon^2}}\right)^2}\\ &=\frac{\sqrt{\sigma_x^2+\sigma_\epsilon^2}}{\sigma_x\sigma_\epsilon\sqrt{2\pi}}e^{-\frac{1}{2}\left(\left(\frac{y-x}{\sigma_\epsilon}\right)^2+\left(\frac{x-\mu_x}{\sigma_x}\right)^2-\left(\frac{y-\mu_y}{\sqrt{\sigma_x^2+\sigma_\epsilon^2}}\right)^2\right)} \end{align*}

Is this a normal distribution? I tried expanding the quadratics inside the exponential and combining the fractions with common denominator $\sigma_x^2\sigma_\epsilon^2(\sigma_x^2+\sigma_\epsilon^2)$. Below, I've written how the numerator expands (utilizing $\mu_y=\mu_x$) and cancelling some terms:

\begin{align*} (y-2xy+x^2)(\sigma_x^4+\sigma_x^2\sigma_\epsilon^2)+(x-2\mu_x x + \mu_x^2)(\sigma_\epsilon^2\sigma_x^2+\sigma_\epsilon^4)-(y^2-2\mu_y y + \mu_y^2)(\sigma_x^2\sigma_\epsilon^2)\\ = y^2\sigma_x^4-2xy\sigma_x^4-2xy\sigma_\epsilon^2\sigma_x^2+x^2\sigma_x^4 + x^2\sigma_\epsilon^2\sigma_x^2+x^2\sigma_\epsilon^4+x^2\sigma_\epsilon^2\sigma_x^2-2\mu_x x \sigma_\epsilon^4-2\mu_x x \sigma_\epsilon^2\sigma_x^2 + \mu_x \sigma_\epsilon^2 +2\mu_x y \sigma_x^2\sigma_\epsilon^2 \end{align*}

I'm was hoping for more cancellation, so I'm not sure how I could factor this.

Is there a better way to find the conditional distribution $x|y$, and maybe to verify that it is a normal distribution?

$\endgroup$
1
  • 1
    $\begingroup$ Complete the square inside the exponent in terms of $x$, i.e. write it as $a(x-x_0)^2 +b$, and you will get a normal distribution $\endgroup$
    – J. Delaney
    Jun 18, 2022 at 16:24

1 Answer 1

0
$\begingroup$

Thanks to @j-delaney for the hint on completing the square.

We have

\begin{align*} f(x|y)&=\frac{\sqrt{\sigma_x^2+\sigma_\epsilon^2}}{\sigma_x\sigma_\epsilon\sqrt{2\pi}}e^{-\frac{1}{2}\left(\left(\frac{y-x}{\sigma_\epsilon}\right)^2+\left(\frac{x-\mu_x}{\sigma_x}\right)^2-\left(\frac{y-\mu_y}{\sqrt{\sigma_x^2+\sigma_\epsilon^2}}\right)^2\right)}\\ \end{align*}

Consider just the argument of the exponential, and ignore the factor of $-1/2$ for now.

\begin{align*} A&=\left(\frac{y-x}{\sigma_\epsilon}\right)^2+\left(\frac{x-\mu_x}{\sigma_x}\right)^2-\left(\frac{y-\mu_y}{\sqrt{\sigma_x^2+\sigma_\epsilon^2}}\right)^2\\ &=\frac{(\sigma_x^2+\sigma_\epsilon^2)x^2-2(\sigma_x^2y+\sigma_\epsilon^2\mu_x)x+\sigma_x^2y^2+\sigma_\epsilon^2\mu_x^2}{\sigma_x^2\sigma_\epsilon^2}-\left(\frac{y-\mu_y}{\sqrt{\sigma_x^2+\sigma_\epsilon^2}}\right)^2\\ &=\text{complete the square and simplify}...\\ &=\frac{\left(\sqrt{\sigma_x^2+\sigma_\epsilon^2}x-\frac{\sigma_x^2y+\sigma_\epsilon^2\mu_x}{\sqrt{\sigma_x^2+\sigma_\epsilon^2}}\right)^2}{\sigma_x^2\sigma_\epsilon^2}+\left(\frac{y-\mu_y}{\sqrt{\sigma_x^2+\sigma_\epsilon^2}}\right)^2-\left(\frac{y-\mu_y}{\sqrt{\sigma_x^2+\sigma_\epsilon^2}}\right)^2\\ &=\left(\frac{x-\frac{\sigma_x^2y+\sigma_\epsilon^2\mu_x}{\sigma_x^2+\sigma_\epsilon^2}}{\frac{\sigma_x\sigma_\epsilon}{\sqrt{\sigma_x^2+\sigma_\epsilon^2}}}\right)^2 \end{align*}

So the entire PDF of $x|y$ can be written

$$ f(x|y)=\frac{1}{\frac{\sigma_x\sigma_\epsilon}{\sqrt{\sigma_x^2+\sigma_\epsilon^2}}\sqrt{2\pi}}e^{-\frac{1}{2}\left(\frac{x-\frac{\sigma_x^2y+\sigma_\epsilon^2\mu_x}{\sigma_x^2+\sigma_\epsilon^2}}{\frac{\sigma_x\sigma_\epsilon}{\sqrt{\sigma_x^2+\sigma_\epsilon^2}}}\right)^2} $$

which implies $$x|y \sim N(\mu_{x|y},\sigma_{x|y}^2)$$ with \begin{align*} \mu_{x|y}&=\frac{\sigma_x^2y+\sigma_\epsilon^2\mu_x}{\sigma_x^2+\sigma_\epsilon^2}\\ \sigma_{x|y}&=\frac{\sigma_x\sigma_\epsilon}{\sqrt{\sigma_x^2+\sigma_\epsilon^2}} \end{align*}

The implication of this distribution could be interpreted as the answer to the question "How should I estimate the true value of $x$ given the measurement $y$? The most likely value or best estimation of $x$ given $y$ is the expected value $E(x|y)=\mu_{x|y}$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.