4
$\begingroup$

Sorry for my simple doubt, but I'm quite newbie and don't clearly get how to sample the posterior Bayesian distribution. My likelihood and prior are normal and I know how to calculate the posterior. How do I sample now? Can someone explain it in the simplest way?


EDIT:

Bayes theorem states: $$ p(θ|y)=\frac{p(y|θ)p(θ)}{p(y)} $$

  1. When I am sampling, what example do I sample for the posterior?

  2. I read that the rnorm command (in R) can easily sample; I gather the syntax would be: rnorm(n samples, mean, std). Is this kind of sampling as valid as MCMC? What is it executing there exactly?

  3. Is sampling from a normal posterior equal to sampling from an isolated normal distribution?

$\endgroup$
  • 1
    $\begingroup$ Welcome to our site! It isn't really clear what you're asking here at the moment, I think you are going to have to be rather more specific. If you are just looking for R code, note that questions asking how something is done in a particular language or software package are off-topic here - have a look at our help center. $\endgroup$ – Silverfish Apr 16 '16 at 21:23
  • $\begingroup$ I agree with @Silverfish. This question looks about R code which is more suitable for Stack Overflow. The posterior that you have obtained should also be a Normal distribution, so you can use rnorm in R. $\endgroup$ – Greenparker Apr 16 '16 at 21:31
  • $\begingroup$ @Greenparker Without a reproducible example, I don't think the question in its current state is well-suited for SO either. $\endgroup$ – Silverfish Apr 16 '16 at 21:37
  • 2
    $\begingroup$ I can imagine this having enough statistical content to be on topic here. Can you say more about your situation, your data & your model? At present, I'm not sure if there's much here that can be answered. $\endgroup$ – gung - Reinstate Monica Apr 16 '16 at 22:09
  • 2
    $\begingroup$ I think this has enough statistical content to be reopened now. $\endgroup$ – gung - Reinstate Monica Apr 17 '16 at 13:18
2
$\begingroup$

I read that the rnorm command (in R) can easily sample; I gather the syntax would be: rnorm(n samples, mean, std). Is this kind of sampling as valid as MCMC? What is it executing there exactly?

In principle, sampling form a posterior distribution is the same as sampling from any other distribution. If the distribution is of a known form (like a Gaussian, Gamma, Inverse Gamma, Beta etc), then you can use inbuilt functions in R to sample from these.

These functions like rnorm use iid sampling techniques, and not MCMC. This is a good thing. MCMC produces correlated samples that are not identically distributed. MCMC is used when iid sampling techniques fail. Unfortunately, in many Bayesian settings, the posterior distribution is not a known distribution, and of a complicated form. Sampling from such a distribution often requires MCMC.

However, known posterior distributions are sampled from using inverse CDF, variable transformation or Rejection sampling techniques which provide iid samples. I am not sure what rnorm uses, but one of the most common ways of sampling from a Normal distribution is the Box-Mueller transformation.

So in short, if your posterior distribution is of a known form, and there exists a function to sample from it, go ahead and use that function.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.