Non-conjugate improper uniform priors in Bayesian data analysis: how to handle infinite sums?

I've been working on Griddy Gibbs sampler (paper: Ritter and Tanner) and I've implemented it in R. But I've faced a problem when I started thinking on its uses in other contexts.

If I try to use an impoper uniform prior in a non conjugate setting, I'm supposed to perform numerical integration with the lower and upper bounds of the priors.

When using flat priors for continuous values, the upper and lower values are simply minus and plus infinity. Thus, it is not possible to have a grid approximation to this interval.

Are there any methods to handle this configuration? So far, I've been happy with the results of my sampler, but this feels like a situation that I can't handle with my current method.

• I'm basically talking about the case where the domain expert or the system that is using the model basically has no idea at all about the hyperparameter. A~N(mu,sigma) -> there is no idea about the value of mu. Mar 12, 2012 at 14:28

First of all, it doesn't matter whether you use improper uniform prior or just very diffuse gaussian distribution as your prior, you will end up with infinite bounds. However, you have to think about your posterior when programming Griddy-Gibs sampler: you have to make sure that you posterior is proper, since improper priors sometimes could lead to improper posteriors. Theoretically, this posterior could range from $-\infty$ to $\infty$. But whole its mass will be concentrated in some finite interval, e.g. if your posterior is standard normal distribution ($\mu =0, \sigma =1$), then almost all its mass will be concentrated in the interval $[-3,3]$. So, you don't have to make infinite grid. Just use some appropriate interval and if you will see that posterior distribution is "cut", then widen your interval and refine your grid.