In Bayesian models, can you use Uniform(-inf, inf) as a prior?

Question

In Bayesian models, can you use Uniform(-inf, inf) as a prior?

I ask because in an class, we looked at MH MCMC sampler, and showed that to sample from a distribution, we need not explicitly solve for the denominator because the numerator is proportional to the the posterior, which will inform the sampler where it should spend more/less of it's time, so you really only need be concerned with the prior and likelihood terms.

I asked the question, "what if the prior term in the numerator was just multiplying the likelihood by 1?" To which my professor said, "this would be analogous to specifying a Uniform prior with support from negative infinity to positive infinity, as there are no range limitations and every value that the parameter could take on would be weighted the same."

First, I'm not sure whether this is or isn't okay. And second, I've heard that 'there really aren't any uninformative priors' though this sounds about as uninformative as a prior could get.

Could someone clarify?

Answer 1

On this forum, there are a lot of related questions and answers about flat priors, like the ones above. They are not uniform priors because they are not distributions but $\sigma$-finite measures (with infinite mass) and they are not the most uninformative or non-informative priors for many reasons detailed in these answers (and Bayesian textbooks). If the posterior attached to the likelihood $f(x|\theta)$ and a flat (constant) prior $\pi(\theta)=c$ is well-defined, ie can be normalised into a probability density for almost all realisations of the random variable $X$ behind the observed data, $$\int_\Theta f(x|\theta)~\text d\theta < \infty\qquad\forall x\quad\text{a.s.}$$ then using this extension of the standard Bayesian framework is acceptable.

Note: the question is unrelated to MCMC (although one should not use MCMC with an improper posterior). The proper entry keyword is improper priors which is a section or a chapter of all Bayesian textbooks. Improper priors are $\sigma$-finite measures $\pi(\cdot)$ (with infinite mass) that can be used as prior measures provided $$\int_\Theta f(x|\theta) \pi(~\text d\theta) < \infty\qquad\forall x\quad\text{a.s.}$$ A flat prior (over an unbounded space) is a particular case of improper prior but not a very special one since a flat prior does not stay constant under most reparameterisations (changes of variables).

Answer 2

If parameter estimation is all you're interested in, then yes, you can use (an improper prior such as) a uniform prior, and Jeffreys (1961, Theory of probability, Clarendon Press) frequently does so in that context. However, any model with an improper prior over a parameter will have a big problem when it comes to model comparison: computing the marginal likelihood therein involves dividing through by the infinitely large partition function of the improper prior. As a result, the marginal likelihood in, and therefore the posterior probability of, the model with the improper prior will be smaller by an infinite factor than the marginal likelihood in, and posterior probability of, any model with a proper prior.