I would like to know the mathematical calculation step by step processes with beta prior and binomial likelihood for posterior predictive distribution.

enter image description here

  • 1
    $\begingroup$ Since this sounds like a self-study question, please add the tag and let us know where you get stuck between the definition of a predictive [start] and the value of this predictive [end]? $\endgroup$ – Xi'an Nov 25 '19 at 15:04
  • 2
    $\begingroup$ Is this homework? If so, please add the self-study tag and show us what you've tried and where you're stuck, and we'll help you through the difficult parts. $\endgroup$ – jbowman Nov 25 '19 at 15:04

Posterior predictive distribution is original pdf times posterior. It's your predictions on 'more' data, given your current data. But your current data is itself dependent on your prior. So really, it's your predicted 'more' data given you've worked out the posterior. Useful link.

Binomial with beta prior is a conjugate prior with parameters found here

To find the posterior, just group multiply the likelihood of data with prior. For example, Posterior of Binomial with Beta Prior, .i.e. the parameter p of the binomial distribution is itself distributed on the beta distribution, is calculated as:

$\prod_{i=1}^{N}$ ${N}\choose{x_i}$ $p^{x_i}(1-p)^{n-x_i}$ $ \frac{1}{B(\alpha,\beta)}p^{\alpha-1}(1-p)^{\beta-1}$

All terms that do not contain p, which is the parameter that is distributed in question, are just constants you can just find the distribution that is proportional to whatever that product equals

$\propto p^{\sum_{i=1}^{N} x_i}p^{\alpha-1}(1-p)^{\sum_{i=1}^{N} n_i-x_i}(1-p)^{\beta-1}$

$\propto p^{(\alpha+\sum_{i=1}^{N} x_i)-1}(1-p)^{(\beta+\sum_{i=1}^{N} n_i-x_i)-1}$

So, posterior distirubtion is propotional to a Beta distribution with parameters, $\alpha=\alpha+\sum_{i=1}^{N} x_i$ and $\beta=\beta+\sum_{i=1}^{N} n_i-x_i$

Put that as your new prior and times it with your original pdf, except this time you full have to do the integral rather than just saying it's 'proportional to' .i.e. The trick will be to kick everything not related to the parameter out of the intgeral, group the exponents to find out what you have multiply with to cancel the intgeral, find a way to multiply by '1' to get a pdf to cancel the integral.

$\int_{0}^{1}$ ${n}\choose{x}$$p^x(1-p)^{n-x}$ $\frac{1}{B(\alpha+\sum x_i,\beta+\sum n_i-\sum x_i)}$ $p^{\alpha+\sum x_i-1}(1-p)^{\beta+\sum n_i-\sum x_i-1}dp$

kick everything not p out

${n}\choose{x}$$\frac{1}{B(\alpha+\sum x_i,\beta+\sum n_i-\sum x_i)}$ $\int_{0}^{1}$$p^x(1-p)^{n-x}$$p^{\alpha+\sum x_i-1}(1-p)^{\beta+\sum n_i-\sum x_i-1}dp$

Group the exponents in the integral.

$\int_{0}^{1}$$p^{x+\alpha+\sum x_i-1}$ $(1-p)^{n-x+\beta+\sum n_i-\sum x_i-1}dp$

Do a trick where you multiply by '1' in the form of $\frac{B(x+\alpha+\sum x_i,n-x+\beta+\sum n_i-\sum x_i)}{B(x+\alpha+\sum x_i,n-x+\beta+\sum n_i-\sum x_i)} $

The inside will resolve to another beta distribution and hence integrating over its range is simply 1.

So that just leaves the outside. ${n}\choose{x}$$\frac{B(x+\alpha+\sum x_i,n-x+\beta+\sum n_i-\sum x_i)}{B(\alpha+\sum x_i,\beta+\sum n_i-\sum x_i)}$

This is kind of cheating but that function is a pmf--the beta-binomial pmf where $n=n$, $k=x$ , $\alpha=\alpha+\sum x_i$ , $\beta=\beta+\sum n_i-\sum x_i$

  • 1
    $\begingroup$ No the predictive dist gives you the distribution of your unobserved/'more' data given what you have observed. Imagine I wanted to sample my unobserved data. Imagine that so far I've been sampling from a binomial, i can keep doing that, or I can make an assumption that p is itself distributed on a beta distribution, if my assumption is correct, I now have a better dist to sample from. $\endgroup$ – Huy Pham Nov 25 '19 at 18:48
  • 1
    $\begingroup$ To find the predictive dist, set up that integral between your data's dist and the dist of the posterior you want and marginalize out the posterior, i.e. integrate with respect to the parameter and over the range of the parameter. en.wikipedia.org/wiki/Posterior_predictive_distribution If the integral doesn't have a closed form solution (most of them won't; the example you wanted was a lucky exception)--you'll need to use sampling methods using winBUGS or stan. $\endgroup$ – Huy Pham Nov 25 '19 at 18:50
  • 1
    $\begingroup$ I'm sorry but I feel you might be mistaking what I'm saying. I watched the video, and he's using the posterior predictive distribution to find the probability x=1 i.e. person is diseased. In that specific case it turns out that the expectation of the beta binomial is exactly that answer. But the integral doesn't equal to an expected value--it's a distribution, it's a function that takes in values and spits out the probabilities that value will occur. $\endgroup$ – Huy Pham Nov 26 '19 at 8:16
  • 1
    $\begingroup$ In answer to your questions, x is number of heads from the original sample used to get the posterior. n is from the distribution of the 'new'/unobserved data, although you will want to set that as the same n as the original distribution anyway. So eg. You can now predict the prob that there will be 3 heads (x=3) out of 10 tosses (n=10) for new data, given that your observed data where the probability of heads was distributed beta (a,b). x that is used to calculate the expectation is the old x, but you can put new x into the beta binomial to get its prob. n you can just set basically. $\endgroup$ – Huy Pham Nov 26 '19 at 8:17
  • 1
    $\begingroup$ Ok that’s about as far as I understand before I start talking about things I don’t know. But my answer for how to calculate the pred dist should be roughly right. Even if the calculations are slightly wrong, the overall idea of integrating and the trick of kicking out constants and dividing by ‘1’ is how you do it. $\endgroup$ – Huy Pham Nov 26 '19 at 8:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.