2
$\begingroup$

I've been working on Griddy Gibbs sampler (paper: Ritter and Tanner) and I've implemented it in R. But I've faced a problem when I started thinking on its uses in other contexts.

If I try to use an impoper uniform prior in a non conjugate setting, I'm supposed to perform numerical integration with the lower and upper bounds of the priors.

When using flat priors for continuous values, the upper and lower values are simply minus and plus infinity. Thus, it is not possible to have a grid approximation to this interval.

Are there any methods to handle this configuration? So far, I've been happy with the results of my sampler, but this feels like a situation that I can't handle with my current method.

$\endgroup$
  • $\begingroup$ I'm basically talking about the case where the domain expert or the system that is using the model basically has no idea at all about the hyperparameter. A~N(mu,sigma) -> there is no idea about the value of mu. $\endgroup$ – mahonya Mar 12 '12 at 14:28
2
$\begingroup$

First of all, it doesn't matter whether you use improper uniform prior or just very diffuse gaussian distribution as your prior, you will end up with infinite bounds. However, you have to think about your posterior when programming Griddy-Gibs sampler: you have to make sure that you posterior is proper, since improper priors sometimes could lead to improper posteriors. Theoretically, this posterior could range from $-\infty$ to $\infty$. But whole its mass will be concentrated in some finite interval, e.g. if your posterior is standard normal distribution ($\mu =0, \sigma =1 $), then almost all its mass will be concentrated in the interval $[-3,3]$. So, you don't have to make infinite grid. Just use some appropriate interval and if you will see that posterior distribution is "cut", then widen your interval and refine your grid.

$\endgroup$
  • 1
    $\begingroup$ It's sometimes useful to remember Peter Huber's comment to the effect of: a standard Cauchy distribution truncated at +/- 1 million is indistinguishable, practically speaking, from a standard Cauchy distribution, but all its moments are finite. $\endgroup$ – jbowman Mar 12 '12 at 14:10
  • $\begingroup$ Thanks, I've edited the question to make it clear that I'm referring to improper uniform prior. What would be the preferred method of checking if the posterior is proper? This is a quite common warning in most texts, but I have not seen a particular method for checking properness. Instinct makes me think about re-normalising the posterior, am I being too naive? $\endgroup$ – mahonya Mar 12 '12 at 14:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.