# Bayesian inference with Gaussian distributions

This is Problem 4(c), Chapter 2 from Thrun's Probabilistic Robotics . Note that this is self-study and not homework.

Suppose I know my position $x$ to be a normal distribution with density $N(\mu_1=1000,\sigma_1^2=900)$. I query a position sensor, which outputs a measurement $z=1100$; however this sensor is faulty, and its output is a normal distribution with variance $\sigma_2^2=100$.

Given the prior probability and sensor error, how likely is it that the measurement was the given value,$z$ = $1100$?

Given that $p(x) = N(\mu_1,\sigma_1)$ and $p(z|x) = N(\mu_2=1100,\sigma_2)$, the updated position is given by Bayes' rule:

$p(x|z) = \frac{N(\mu_1,\sigma_1)N(\mu_2,\sigma_2)}{\int_{\mathbb{R}}N(\mu_1,\sigma_1)N(\mu_2,\sigma_2)dx}$

I found that $p(x|z) = N(\mu=1090,\sigma^2=90)$. But I am unable to take this reasoning further, and am unclear of whether I made any progress. Can you help?

I think the question as described does not require the calculation of the posterior. First, note that technically, the question asks for the likelihood of a point value under a continuous distribution (with bounded PDFs). This likelihood is trivially 0 for all such point values. Rather, I would interpret the question as follows.

Let $O$ be a random variable describing your observed position, and let $X$ be a random variable describing your actual position. I think the question is then:

What is $P( O = 1100 )$ (the probability of observing our position to be $1100$) when we know that for any position $x$ the probability of being there is $P( X = x ) = \mathcal{N}(x \mid \mu = 1000, \sigma^2 = 900)$ and the probability of observing position $o$ when we are at position $x$ is $P( O = o \mid X = x ) = \mathcal{N}( o \mid \mu = x, \sigma^2 = 100 )$. The answer is: $$P( O = 1100 ) = \int_{-\infty}^{\infty} P( O = 1100 \mid X = x ) P( X = x ) \,\text{d}x \enspace,$$ which equals \begin{align*} P( O = 1100 ) & = \int_{-\infty}^{\infty} \mathcal{N}\left( 1100 \,\middle|\, \mu_1 = x, \sigma_1^2 = 100 \right) \mathcal{N}\left( x \,\middle|\, \mu_2 = 1000, \sigma_2^2 = 900 \right)\,\text{d}x \\ & = \int_{-\infty}^{\infty} \mathcal{N}\left( x \,\middle|\, \mu_1 = 1100, \sigma_1^2 = 100 \right) \mathcal{N}\left( x \,\middle|\, \mu_2 = 1000, \sigma_2^2 = 900 \right)\,\text{d}x \enspace, \end{align*} where I used the symmetry of the normal distribution to move $x$ to the argument rather than the mean in the first PDF. In general, the product of two normal PDFs is a scaled normal PDF, defined by $$\mathcal{N}\left( x \,\middle|\, \mu_1, \sigma_1^2 \right) \mathcal{N}\left( x \,\middle|\, \mu_2, \sigma_2^2 \right) = \mathcal{N}\left( x \,\middle|\, \frac{\mu_1 \sigma_2^2 + \mu_2 \sigma_1^2}{\sigma_1^2 + \sigma_2^2}, \frac{\sigma_1^2 \sigma_2^2}{\sigma_1^2 + \sigma_2^2 } \right) \mathcal{N}\left( \mu_1 \,\middle|\, \mu_2, \sigma_1^2 + \sigma_2^2 \right) \enspace.$$ (It's an interesting exercise to check this: write out the product and complete the square in the exponent.)

This implies the factor in the intergral is just a scaled normal PDF, whose integral equals \begin{align*} & \int_{-\infty}^{\infty} \mathcal{N}\left( x \,\middle|\, \frac{\mu_1 \sigma_2^2 + \mu_2 \sigma_1^2}{\sigma_1^2 + \sigma_2^2}, \frac{\sigma_1^2 \sigma_2^2}{\sigma_1^2 + \sigma_2^2 } \right) \mathcal{N}\left( \mu_1 \,\middle|\, \mu_2, \sigma_1^2 + \sigma_2^2 \right) \,\text{d}x \\ & = \mathcal{N}\left( \mu_1 \,\middle|\, \mu_2, \sigma_1^2 + \sigma_2^2 \right) \int_{-\infty}^{\infty} \mathcal{N}\left( x \,\middle|\, \frac{\mu_1 \sigma_2^2 + \mu_2 \sigma_1^2}{\sigma_1^2 + \sigma_2^2}, \frac{\sigma_1^2 \sigma_2^2}{\sigma_1^2 + \sigma_2^2 } \right) \,\text{d}x \\ & = \mathcal{N}\left( \mu_1 \,\middle|\, \mu_2, \sigma_1^2 + \sigma_2^2 \right) \\ & = \mathcal{N}\left( 1100 \,\middle|\, \mu = 1000, \sigma^2 = 1000 \right) \\ & \approx 8.5 \cdot 10^{-5} \enspace. \end{align*} Perhaps more interesting is finding a value of at least 1100, which would equal $$P( O \geq 1100 ) = \int_{1100}^{\infty} \mathcal{N}\left( 1100 \,\middle|\, \mu = 1000, \sigma^2 = 1000 \right) \approx 7.8 \cdot 10^{-4} \enspace.$$ This tells us that the observed value is in fact pretty unlikely.

Another, perhaps friendlier, way to find the same answer is to note that we are looking for the probability $P( X + Y = 1100 )$, where $X$ is a normal random variable with mean $\mu_1=1000$ and variance $\sigma_1^2=900$ and $Y$ is a normal random variable with mean $\mu_2 = 0$ and variance $\sigma_2^2=100$. For any two independent normal random variables, their sum is another normal random variable with mean $\mu = \mu_1 + \mu_2$ and variance $\sigma^2 = \sigma_1^2 + \sigma_2^2$.

• MLS , thanks for the answer. FYI, I already know how to complete the square for the integral. – PKG Mar 12 '13 at 17:55

The next step I would do is to check if the value z=1100 is among the credible interval for your posterior distribution. For this you can check if it is within the 95% HDI (highest density interval) of your distribution. If it's not, than you can say that z=1110 is not a credible value for your posterior (analogous to frequentist reject hypothesis).

Doing pnorm(q=1100,mean=1090,sd=sqrt(90),lower.tail=FALSE) in R yields 0.1459203 which is bigger than (1-0.95)/2=0.025, so we can say that value z=1100 is within the 95% HDI and therefore it is a credible value. (The accumulated distribution can be used this way to get the HDI because of the concavity and symmetry of the normal distribution).

After doing that, if it does make sense to your problem, you might also try to establish a ROPE (region of practical equivalence) and check if your the posterior 95% HDI is in entirely inside it (analogous to frequentist accept hypothesis, if such thing exist).