1
$\begingroup$

I'm looking to better understand how Bayesian posterior distributions can be derived from different types of data. There are plenty of examples out there showing how a posterior can be derived from a beta prior and Bernoulli trials as data/likelihood. What I want to understand is how this method can be extended to other scenarios.

Example:

  1. Let's say we have some discrete data that follows a weibull distribution with the parameters of shape=35 and scale=282 (I know I'm bending the rules a bit here, but it was the easiest way to generate the data.)
  2. Our prior is a beta distribution with alpha and beta both equal to zero... a completely uniformed prior.
  3. How can a posterior distribution be derived from this data?
  4. What if the prior changes to a beta distribution of alpha=3 and beta=4 representing a change in belief?

I've included some R code to help generate the data I'm describing:

#Example data 
sims <- round(rweibull(10000, shape = sp, scale = sc),0).

#Example beta dist
library(manipulate)

x <- seq(0, 1, length = 50)
manipulate(plot(x,dbeta(x, alp, bet),type="l",lwd=2,col="lightblue",xlim=c(0,1),breaks=20), 
           alp= slider(0,100), 
           bet= slider(0,100)) 

Any resources would be much appreciated, as would examples using JAGS/Stan if at all possible. I know I made some pretty broad assumptions about using a beta distro as prior as well, so if there is a better approach please let me know.

Thanks in advance!

$\endgroup$
  • $\begingroup$ Your example is...pretty whacky, so much so that it's hard to parse what you're getting at. A Weibull is continuous, and with positive parameters: It'd be strange to use it for discrete data, and more so to choose a prior with support only on $(0,1)$. The short answer to "how this method can be extended to other scenarios" is "Bayes rule," but I suspect you're looking for something a bit more constructive. $\endgroup$ – Sean Easter Mar 23 '17 at 19:55
  • $\begingroup$ Yep, I know it's a bit of aodd scenario, like how I rounded the weibull data to make it discrete. But to your point my main question is based around how to functionally extend the bayesian methodology away from Bernoulli trials to other distributions/scenarios. $\endgroup$ – DaBenski Mar 24 '17 at 14:53
  • $\begingroup$ Understood, I'll post an answer when I'm on desktop (currently mobile). $\endgroup$ – Sean Easter Mar 24 '17 at 15:44
1
$\begingroup$

There's really no magic to "extending" those examples to other data types: Simply choose a relevant likelihood and a suitable prior. So, to model discrete data:

  • Pick a likelihood $p(x|\theta)$ that generates discrete data, e.g. Poisson or negative binomial.
  • Choose a suitable prior $p(\theta)$. It would be peculiar to choose a prior without support on the full domain of the likelihood parameters: You're essentially ruling out whole intervals of possible parameters. (Hence why beta-Weibull is odd.)
  • Either approximate (MCMC, variational) or analytically solve for the posterior using Bayes rule: $$p(\theta|x) = \frac{p(x|\theta)p(\theta)}{p(x)}$$

Those Bernoulli examples you run into choose those distributions because beta is conjugate to Bernoulli: The posterior can be solved for analytically. To figure out a simple discrete conjugate model in this vein, start here with one of the distributions mentioned above. In this case, changing your choice of prior parameters is straightforward. (Read about any conjugate prior and you'll generally see the posterior expressed in terms of prior parameters and data.)

For models lacking analytic solutions—ones where $p(x)$ is difficult to compute—you're left to approximate methods. The full role of the prior in these seems beyond the scope of your question.

Interestingly enough, your rounded Weibull gives this probability mass function for an integer $x$. (Follows from the CDF of the Weibull.)

$$p(x|\lambda, k) = \exp{-\Big(\frac{x+\frac{1}{2}}{\lambda}}\Big)^k - \exp{-\Big(\frac{\max{\big(x-\frac{1}{2}},0\big)}{\lambda}}\Big)^k$$

Maybe you want to put a prior on either of those parameters and see what you can come up with analytically or via MCMC. Could be fun.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.