2
$\begingroup$

I have quite a newbie doubt about Bayesian inference. Let's say that my prior data is composed by a Gaussian distribution (mean1, standard deviation1).

My likelihood is another Gaussian with mean2, std deviation2.

Now, my question is how can I get the posterior, please? The most clear tutorial I found is this, in eq(28), but I do not have clear whether the author is applying Monte Carlo there or not.

$\endgroup$
1
2
$\begingroup$

Finding the posterior distribution does not involve Monte Carlo, but involves the Bayes rule. If the posterior turns out to be complicated (usually meaning that it is not a known distribution), then Monte Carlo methods are used to sample from the posterior. Thus, the first step is to always try and write down the posterior.

Notationally, your likelihood is $Y_i|\mu_1 \sim N(\mu_2, \sigma_2^2)$ assuming $\sigma_2^2 > 0$ is known. Let the prior on $\mu_2$ be $N(\mu_1, \sigma_1^2)$ where $\mu_1 \in \mathbb{R}, \sigma_1^2 > 0$ are fixed.

The posterior distribution is found by the Bayes Rule.

$$f(\mu_2|y) = \dfrac{f(\mu_2) \prod_{i=1}^{n}f(y_i|\mu_2)}{\prod_{i=1}^{n}f(y_i|\mu_2))}. $$

Since we condition on $y$, $f(y_i)$ is a constant.

\begin{align*} f(\mu_2|y) & \propto f(y|\mu_2)f(\mu_2)\\ &= \prod_{i=1}^{n} \left(\frac{1}{\sqrt{2\pi \sigma_2^2}} \exp \left\{-\dfrac{(y_i - \mu_2)^2}{2\sigma_2^2} \right\} \right) \frac{1}{\sqrt{2\pi \sigma_1^2}} \exp \left\{-\dfrac{(\mu_2 - \mu_1)^2}{2\sigma_1^2} \right\}\\ & \vdots \end{align*}

If you keep solving this like the link you shared, you will see that this takes the form of a Normal distribution in $\mu_2$ with the parameters as indicated in the link. You can find more details in the following links

https://www.youtube.com/watch?v=c-d05z0_5mw http://www.people.fas.harvard.edu/~plam/teaching/methods/conjugacy/conjugacy_print.pdf

$\endgroup$
3
  • $\begingroup$ Thanks for the fast reply. OK, and what is the n that first appears on eq (16). Have you got a numerical example of this? Thx $\endgroup$
    – galtor
    Mar 27 '16 at 15:33
  • $\begingroup$ If you expand the product, you will see the $n$ appear. It is just a matter of going through the steps of algebra. $\endgroup$ Mar 27 '16 at 15:35
  • $\begingroup$ And finally, please: if I want to make a prediction (sport prediction or poll prediction) what kind of posterior must I use: predictivie posterior or simply posterior? $\endgroup$
    – galtor
    Mar 27 '16 at 15:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.