Sampling from an Improper Distribution (using MCMC and otherwise)

Question

My basic question is: how would you sample from an improper distribution? Does it even make sense to sample from an improper distribution?

Xi'an's comment here kind of addresses the question, but I was looking for some more details on this.

More specific to MCMC:

In talking about MCMC and reading papers, authors stress on having obtained proper posterior distributions. There is the famous Geyer(1992) paper where the author forgot to check if their posterior was proper (otherwise an excellent paper).

But, suppose a we have a likelihood $f(x|\theta)$ and an improper prior distribution on $\theta$ such that the resulting posterior is also improper, and MCMC is used to sample from the distribution. In this case, what does the sample indicate? Is there any useful information in this sample? I am aware that the Markov chain here is then either transient or null-recurrent. Are there any positive take-aways if it is null-recurrent?

Finally, in Neil G's answer here, he mention's

you can typically sample (using MCMC) from the posterior even if it's improper.

He mentions such sampling is common in deep learning. If this is true, how does this make sense?

Answer 1

Sampling from an improper posterior (density) $f$ does not make sense from a probabilistic/theoretical point of view. The reason for this is that the function $f$ does not have a finite integral over the parameter space and, consequently, cannot be linked to a (finite measure) probability model $(\Omega,\sigma,{\mathbb P})$ (space, sigma-algebra, probability measure).

If you have a model with an improper prior that leads to an improper posterior, in many cases you can still sample from it using MCMC, for instance Metropolis-Hastings, and the "posterior samples" may look reasonable. This looks intriguing and paradoxical at first glance. However, the reason for this is that MCMC methods are restricted to numerical limitations of the computers in practice, and therefore, all supports are bounded (and discrete!) for a computer. Then, under those restrictions (boundedness and discreteness) the posterior is actually proper in most cases.

There is a great reference by Hobert and Casella that presents an example (of a slightly different nature) where you can construct a Gibbs sampler for a posterior, the posterior samples look perfectly reasonable, but the posterior is improper!

http://www.jstor.org/stable/2291572

A similar example has recently appeared here. In fact, Hobert and Casella warns the reader that MCMC methods cannot be used to detect impropriety of the posterior and that this has to be checked separately before implementing any MCMC methods. In summary:

1. Some MCMC samplers, such as Metropolis-Hastings, can (but shouldn't) be used to sample from an improper posterior since the computer bounds and dicretizes the parameter space. Only if you have huge samples, you may be able to observe some strange things. How well you can detect these issues also depends on the "instrumental" distribution employed in your sampler. The latter point requires a more extensive discussion, so I prefer to leave it here.
2. (Hobert and Casella). The fact that you can construct a Gibbs sampler (conditional model) for a model with an improper prior does not imply that the posterior (joint model) is proper.
3. A formal probabilistic interpretation of the posterior samples require the propriety of the posterior. Convergence results and proofs are established only for proper probability distributions/measures.

P.S. (a bit tongue in cheek): Do not always believe what people do in Machine Learning. As Prof. Brian Ripley said: "machine learning is statistics minus any checking of models and assumptions".

Answer 2

Giving an alternative, more applied, view from Rod's excellent answer above -

In many, if not most, cases, impropriety of the posterior is a result of choices made for convenience, not a true "I'm absolutely certain of my likelihood function and prior distribution, and look what happened!" effect. Given this, we shouldn't take impropriety too seriously in our applied work unless it's going to mess up our computations. As someone famous (Huber? Tukey?) once observed, in a different context, the difference between a standard Cauchy and a Cauchy truncated at $+/- 10^{100}$ is undetectable, but one has no moments and the other has moments of all orders.

In this context, if I have a posterior distribution for demand for hot dogs at AT&T Park next weekend with upper tail proportional to $1/x$, that's bad news for algorithms that calculate expected values, but if I truncate it at the estimated number of people in San Francisco, a number somewhat larger than the number of hot dogs that will in fact be sold at AT&T park next weekend, all is well, at least in terms of existence of moments. In the latter case, you can think of it as a sort of two-stage application of the real prior - one I use for calculation, which doesn't have an upper bound, and the "extra feature" of it where it's equal to zero above the population of San Francisco...", with the "extra feature" being applied in a step subsequent to the generation of the sample. The real prior is not the one that's used in the MCMC computation (in my example.)

So in principle I would be quite OK with using an MCMC-generated sample from an improper distribution in applied work, but I'd be paying a lot of attention to how that impropriety came about, and how the random sample will be affected by it. Ideally, the random sample wouldn't be affected by it, as in my hot-dog example, where in a reasonable world you'd never actually generate a random number greater than the number of people in San Francisco...

You should also be aware of the fact that your results may be quite sensitive to the feature of the posterior that caused it to be improper, even if you do truncate it at some large number later on (or whatever alteration is appropriate for your model.) You'd like your results to be robust to slight changes that shift your posterior from improper to proper. This can be harder to ensure, but is all part of the larger problem of making sure your results are robust to your assumptions, especially the ones which are made for convenience.