3
$\begingroup$

How should I find the likelihood function of a Bayesian Model?

For example, if I'm given a coin, I can use the Bernoulli Distribution as the likelihood function (because I know in advance that the outputs are 0 or 1 and it fits that distribution).

What if I'm given some data that I don't know the underlying distribution but it seems to follow a gamma distribution? (gamma is just for the example here, it could be normal, exponential, or some weird distribution).

  1. Should I just estimate the parameters of that Gamma (ie fitdist in R) and use these parameters along the likelihood function? (and if that is the case, it wouldn't work with the weird distribution)
  2. Use bayesian modeling to estimate the likelihood function. If that is the case, how can I achieve that? (I'm not expecting a printed code here but more like an explanation that puts me in the right direction)
Improve this question
$\endgroup$
4
$\begingroup$

In practice you never know the likelihood; IMO it's better thought of as a useful model for your data.

In fact, I often find it helpful to replace the words 'prior' and 'likelihood' with 'parameter model' and 'data model' respectively.

If a gamma distribution or what have you seems to model your data well, then it would be a prime choice for a likelihood function.

You could indeed then use maximum likelihood to estimate the parameters - that is, find the parameter values that maximize the likelihood function. You can usually do this via some optimization algorithm if your likelihood is reasonably well-behaved. In R you could use for example optim().

If you want to do a Bayesian treatment you'll want to specify a prior (a parameter model) in addition to your likelihood (your data model). In the case of a $\text{gamma}(\alpha, \beta)$ distribution that means you'd want to specify distributions for $\alpha$ and $\beta$ as well. But you don't usually 'estimate the likelihood function' so to speak; you specify it, and then conduct your analysis around that assumption.

Improve this answer
$\endgroup$
  • $\begingroup$ Great explanation. For the coin flip, the likelihood is clearly a bernoulli. When I define the likelihood, and it doesn't perfectly fit the data - as was the case in this question, would it led me to a wrong posterior? Or the posterior distribution will already cover that likelihood approximation error? $\endgroup$ – Thiago Balbo Sep 11 '15 at 22:21
  • 1
    $\begingroup$ Correct - it could only be 'wrong' in the same sense the likelihood can be 'wrong'. If the likelihood is a poor model for your data then you can expect your ability to do inference (via the posterior) to be pretty limited. $\endgroup$ – jtobin Sep 11 '15 at 23:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.