I have a data which follows gamma distribution and want to know the uncertainty of the parameters of this data.

  • $\text{Data} \sim \text{Gamma} (\alpha, \beta)$
  • Parameters

    $\alpha \sim \text{Gamma} (k_\alpha, \theta_\alpha)$

    $\beta \sim \text{Gamma}(k_\beta, \theta_\beta)$

I used Winbugs (code below).

  for (i in 1:N){
     Y[i] ~ dgamma(k, theta)

  k ~ dgamma(0.1, 0.1)
  theta ~ dgamma(0.1, 0.1)
  1. To plot the likelihood I used a uniform prior then divided the posterior by the prior which makes the posterior same as the likelihood. (Figure1) enter image description here

  2. Next I changed the prior several times and looked what happens, but the problem is when I plotted the likelihood by dividing the posterior by the prior, likelihood changes which shouldn’t, whenever I change the prior. (Figure 2, 3)

enter image description here enter image description here


Is it possible that the likelihood changes when the prior changes? Can somebody help me what the problem is? If the prior is too narrow, is there a possibility that the posterior might be wrong?

  • 4
    $\begingroup$ Likelihood is only a function of the model and the data. If those don't change, the likelihood is the same. $\endgroup$ – Glen_b Mar 26 '14 at 1:07
  • $\begingroup$ Yes, that's why I am confused about the results. The likelihood shouldn't be changed but it moves when the prior changes. $\endgroup$ – moon Mar 26 '14 at 1:12
  • $\begingroup$ Is the posterior a proper distribution? I am wondering if the model evidence is playing a part in this? If the posterior is a proper distribution than for different priors, you have different marginal likelihoods and that should be taken into account to recover the likelihood, perhaps? $\endgroup$ – Luca Mar 26 '14 at 1:14
  • $\begingroup$ Would you please explain about it in detail? I'm not sure about your explanation since I just started to study bayesian. $\endgroup$ – moon Mar 26 '14 at 1:36
  • 3
    $\begingroup$ The posterior is given by $P(\theta|D) = \frac{P(D|\theta)P(\theta)}{\int_{\theta}P(D|\theta)P(\theta)}$. So if you change the prior, the denominator (model evidence) will also change. So, it will not be sufficient to divide the posterior by just $P(\theta)$ to get the likelihood back, you also need to keep track of the evidence term. $\endgroup$ – Luca Mar 26 '14 at 1:54

The likelihood expressed in terms of the prior, the posterior and the evidence is: $$p(D|\alpha,\beta)=\frac{p(\alpha,\beta|D)p(D)}{p(\alpha,\beta)}$$ I assume that what you are looking at is: $$ p(D) \int p(D|\alpha,\beta) \,\mathrm{d}\beta = p(D) \int \frac{p(\alpha,\beta|D)p(D)}{p(\alpha,\beta)} \,\mathrm{d}\beta $$

What you actually should look at is: $$ \frac{\int p(\alpha,\beta|D)p(D) \,\mathrm{d}\beta}{\int p(\alpha,\beta) \,\mathrm{d}\beta} =\frac{ p(\alpha|D) p(D)}{p(\alpha)} = p(D|\alpha) $$

However, without seeing your complete code, we do not know what you are looking at. What we know for sure is that the Likelihood does not depend on the choice of the prior distribution.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.