Fast integration of a posterior distribution

Question

I wish to infer the posterior distribution on the probability of success $\theta$ in some binomial process, the twist being that I know that $\theta$ lies in the interval [0.5, 1].

The trouble is that a Beta distribution that is supported on [0.5, 1] is no longer conjugate. In particular, the prior looks like this ($p$ is the pdf):

$p(\theta) \propto {(\theta-0.5)}^{\alpha-1} {(1-\theta)}^{\beta-1}$

The likelihood, given $x$ successes in $n$ trials, is the usual binomial likelihood:

$p(n,x | \theta) \propto \theta^x {(1-\theta)}^{n-x}$

The posterior, as it stands, looks like this:

$p( \theta | n, x) \propto \theta^x {(\theta-0.5)}^{\alpha-1} {(1-\theta)}^{n-x+\beta-1}$

I would like to be able to make CDF calculations (i.e. compute integrals) of the posterior efficiently. (numerical integration, e.g., may be too slow for my purposes). I am unable to figure out how to do this.

Much appreciate any help the forum can provide.

Answer 1

How accurate does your posterior cdf need to be? You might consider replacing the continuous prior with a discrete approximation:

$p^*(\theta) \propto p(\theta) 1(\theta\in t_1, \dots, t_k)$

where $p(\theta)$ is your original continuous prior.

Then to compute the posterior you just calculate likelihood x prior

$p(\theta|x) \propto p^*(\theta)p(x|\theta)$

over the support of the prior $t_1, \dots, t_k$ and renormalize.

This is called "griddy Gibbs" by some. It can be quite effective if you have an informative prior in which case you can choose the grid points non-uniformly (and, of course, if you can live with a discrete approximation coarse enough to be computationally feasible).

Answer 2

You can always use Monte Carlo Integration or the Midpoint method. With Monte Carlo, you simply generate a bunch of points in your parameter space and see if they are in the area or volume or hyper-dimensional space you are trying to integrate.

From: http://farside.ph.utexas.edu/teaching/329/lectures/node109.html "Let us now consider the so-called Monte-Carlo method for evaluating multi-dimensional integrals. Consider, for example, the evaluation of the area, , enclosed by a curve, . Suppose that the curve lies wholly within some simple domain of area , as illustrated in Fig. 97. Let us generate points which are randomly distributed throughout . Suppose that of these points lie within curve . Our estimate for the area enclosed by the curve is simply" the ratio of random points in the space times the size of the space. The link has a nice picture and a description of the inferior midpoint method that I would suggest skipping.