2
$\begingroup$

I'm reading the book "Bayesian Analysis with Python" and the author provides some python code designed to show the grid search method of obtaining an approximate posterior distribution for the classic coin flipping example. In this example we set a prior on the probability of obtaining a head, then given some data our likelihood is the binomial distribution.

So from my understanding using grid search we will break the interval $[0,1]$ into chunks, in our case 100. We have a prior probability $P(\theta=\theta_{0})$ for each of the points in the discretised grid we multiply this by $P(D|\theta=\theta_{0})$ and we multiply these together to get an un-normalised estimate at each location.

The code is the following

def posterior_grid_approx(grid_points=100, heads=6, tosses=9):
    grid = np.linspace(0, 1, grid_points)
    prior=np.repeat(5,grid_points)
    likelihood=stats.binom.pmf(heads, tosses, grid)
    unstd_posterior = likelihood * prior
    posterior = unstd_posterior / unstd_posterior.sum()    
    return grid, posterior

I don't understand why the prior is set to a list of $5$'s isn't this meant to be a probability value? From my understanding they've broken the grid into 100 different points obtained a binomial pmf for each of these locations and then multiplied the number by 5. How is this a valid prior?

I'd really appreciate if someone could clearly explain the reasoning behind this as i like to understand the code as clearly as the concept. Thanks!

$\endgroup$
  • $\begingroup$ I don't think it makes sense what they do, since the prior should be a PMF and normalized to 1. But, in Bayesian methods you're often calculating a bunch of terms that share a proportionality factor anyways (the evidence probability in the denominator), so you can just absorb the 5's into the proportionality factor and will find the answer to be the same. It would make more sense if the prior were .01 for each point instead of 5. No idea where that number comes from. $\endgroup$ – PrestonH Mar 15 '18 at 3:16
0
$\begingroup$

Sorry about the confusion, the number 5 is arbitrary. You can choose virtually any value you want and you will get the same result. You can think this as multiplying the posterior by a constant. I agree 1 / grid_points will be less confusing.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.