# Return value of uniform distributions for MCMC simulations

I am confused about how what value should be returned from a uniform distribution when using MCMC simulations.

The proper normal distribution is define as $$p(\theta) = \left\{ \begin{array}{cc} 1/(b-a) & \text{if} & a < \theta < b \\ 0 & & \mathrm{otherwise} \end{array}\right.$$

However, in the examples from the Python package I am useing (emcee), they instead use the improve definition $$p(\theta) = \left\{ \begin{array}{cc} 1 & \text{if} & a < \theta < b \\ 0 & & \mathrm{otherwise} \end{array}\right.$$

Now for the parameter estimation itself this makes no obvious difference. Unfortunately I am then using the output from the MCMC algorithm to estimate the marginal likelihood (the evidence). To do this I use the Harmonic mean approximation (which I know is deeply flawed, nevertheless it seems to be doing a reasonable job). In this case the choice of prior makes a significant difference.

Can anyone explain which of these is really appropriate and why? I originally expected the two to reach the same answer, but there is a noticeable difference and it makes me somewhat uncomfortable. Since I am only in the learning stage, I thought it best to ask the stupid questions first.

The uniform PDF is as you say:

$$p(\theta) = \left \{ \begin{array}{ll} 1 / (b-a) \quad &\text{if } a < \theta < b\\ 0 \quad &\text{otherwise.} \end{array} \right.$$

Bayesian methods (MCMC) are usually only concerned with the posterior distributions up to a constant of proportionality, that is:

$$p(\theta | D) \propto p(\theta) p(D | \theta)$$

for some observed data $D$. Thus you only really need the prior distribution up to a constant of proportionality too, which is why you can use:

$$p(\theta) \propto \left \{ \begin{array}{ll} 1 \quad &\text{if } a < \theta < b\\ 0 \quad &\text{otherwise.} \end{array} \right.$$

In which sense the prior on $\theta$ only provides upper ($b$) and lower ($a$) bounds on allowable value of $\theta$. In computing terms, the proportional form is faster.

• So if I keep the constant of proportionality, is my result incorrect or just scaled differently?
– Greg
Apr 2 '15 at 13:19
• @Greg it just does not have a $[0, 1]$ scale so it is not probability proper, but it does not matter if you are not interested in estimating the probabilities.
– Tim
Apr 2 '15 at 13:32
• @Tim I am interested in calculating the probabilities in the end, but by using the harmonic mean approximation. I think I need to do some more reading by the sounds of it, thanks for the help!
– Greg
Apr 2 '15 at 13:57
• @Greg how are you calculating the harmonic mean? I've never done it, but I thought you just use the posterior samples from the MCMC (which you have said are the same for both methods) and calculate 1/Likelihood for each sample (radfordneal.wordpress.com/2008/08/17/…) so the 2 methods should be the same. Perhaps you are seeing the problems with using the harmonic mean approximation.
– Jeff
Apr 2 '15 at 13:58

Bayes theorem states that

$$p(\theta|D) \propto p(D|\theta) p(\theta)$$

so if $\theta$ is constant, and that is the case in both cases you quote, then it does not have any effect on the estimates. Notice that if you want $p(\theta|D)$ to give you proper probabilities (i.e. to integrate to $1$) then the only thing you have to do is to divide by $\int p(D|\theta) p(\theta) d\theta$, that is constant. See also this thread to learn more on normalizing constant in Bayes theorem.