# A question about posterior distribution $p(\mu|\mathcal{D}) = \mu|x_1, \ldots, x_n$

On these course notes, we are given the distribution of the posterior distribution https://people.eecs.berkeley.edu/~jordan/courses/260-spring10/lectures/lecture5.pdf

This famous result can be found in many other places, such as https://www.cs.ubc.ca/~murphyk/Papers/bayesGauss.pdf. However, I am confused about the coefficient (i.e., the $\dfrac{1}{\sqrt{2\pi}\sigma}$) term associated with the Gaussian for the posterior distribution. Here is what I am having trouble with.

We know that given $\mathcal{D} = (x_1, \ldots, x_n), x_i$ iid,

$$p(\mu|\mathcal{D}) \propto p(D|\mu)p(\mu)$$

Suppose that $$p(\mu) = \dfrac{1}{\sqrt{2\pi}\sigma_o} \exp{\dfrac{(x - \mu_o)^2}{2\sigma_o^2}}.$$ and $$p(D|\mu)= \dfrac{1}{(2\pi\sigma^2)^\frac{N}{2}} \exp(-\dfrac{1}{2\sigma^2}\sum\limits_{n = 1}^N (x_n - \mu))$$

then multiplying the expressions together, we obtain:

$$\dfrac{1}{(2\pi\sigma^2)^\frac{n}{2}} \exp{\left[-\dfrac{1}{2\sigma^2}\sum\limits_{n = 1}^N (x_n - \mu)\right]} \dfrac{1}{\sqrt{2\pi}\sigma_o} \exp{\left[\dfrac{(x - \mu_o)^2}{2\sigma_o^2}\right]}$$

While we can perform a complete the square inside of the exponential, what about the constants $\dfrac{1}{(2\pi\sigma^2)^\frac{n}{2}}$ and $\dfrac{1}{\sqrt{2\pi}\sigma_o}$?

I don't see how $\dfrac{1}{(2\pi\sigma^2)^\frac{n}{2}} *\dfrac{1}{\sqrt{2\pi}\sigma_o} = \dfrac{1}{\sqrt{2\pi}\sigma_n^2}$

where $\sigma_n^2 = (1/\sigma_o^2 + n/\sigma^2)^{-1}$ as shown in Lemma 6.

I tried playing around with the terms but I couldn't make them equal, even when $n = 1$. Have I made a mistake or is this because $p(\mu|\mathcal{D})$ is not a "true" probability distribution (i.e., doesn't integrate to 1)?

If so, how would people deal with this leading coefficient during simulation?

Bishop Pattern Reconigition and Machine Learning (2006) Pg. 98

• It's just a constant of integration. Note that symbol "$\propto$" in your initial expression; it means "proportional to", as in "don't worry about the constant of integration until you're done with everything else, then it's whatever is needed to make the posterior integrate to one". Commented Jan 26, 2018 at 4:25
• It is $1/\sqrt{2\pi}\sigma_n$. Commented Jan 26, 2018 at 4:28
• To expand on that... if the constant is not $1/\sqrt{2\pi}\sigma_n$, what it really means is that $p(\mu|D)$ doesn't integrate to 1, which tells you that you have the wrong constant. The kernel / functional form is still that of a Gaussian distribution, though. Commented Jan 26, 2018 at 4:36
• @StackexchangeHouseNinja Did you read jbowman's comments, especially the first one? What do you think "$\propto$" means in $p(\mu \mid D) \propto p(\mu) p(D \mid \mu)$? Commented Jan 26, 2018 at 20:09
• I suspect I see your problem. It appears to me that you think $p(D|\mu)p(\mu)$ is a Normal distribution. It isn't. It's proportional to a Normal distribution. That is why we use the $\propto$ symbol instead of the $=$ symbol in the expression $p(\mu|D) \propto p(D|\mu)p(\mu)$. More specifically, $p(D|\mu)p(\mu) \propto N(\mu_n, \sigma^2_n) = p(\mu|D)$. Commented Jan 27, 2018 at 2:25

So, just to note, $\Pr(\mu|D)$ is a real probability distribution or at least a distribution function. Read lines 8-12 in the commentary of