# Distribution families whose likelihoods integrate to $+\infty$ for some sample values

I've recently started learning about Bayesian statistics, and I came across this very nice answer by Xi'an https://stats.stackexchange.com/a/129908/268693, which [in my slight paraphrasing] says the following: Given a family of distributions $$\{f(\cdot|\theta): \theta \in \Theta \}$$ defined on a sample space $$\mathcal{X}$$, and a prior distribution $$\pi$$, we require that

$$\int_{\Theta} f(x|\theta) \pi(\theta) \,d\theta < \infty \quad \text{ for all } x \in \mathcal{X};$$

otherwise, we do not obtain a valid prior distribution $$\pi(\theta)$$, and so Bayesian inference is not possible. This leads me to the following question:

What are some families of distributions $$\{f(\cdot|\theta): \theta \in \Theta \}$$ that one might encounter in practice for which there exists a set $$E \subset \mathcal{X}$$ of positive measure such that $$\int_{\Theta} f(x|\theta) \,d\theta = + \infty \quad \text{ for all } x \in E?$$

In other words, I'm curious if there is a family of distributions for which a uniform prior leads to an "improper posterior" such that the problem cannot be remedied by re-defining the $$f(\cdot|\theta)$$'s on a set of measure zero.

Here are a couple of examples I came up with:

• The Cauchy distribution: $$f(x|\theta) = \frac{1}{\theta \pi [1 + (x/\theta)^2 ]}, \; x > 0, \; \theta > 0$$. In this case, $$\int_{0}^{\infty} f(x|\theta) \,d\theta = + \infty$$ for all $$x > 0$$.

• A rather contrived example: For each $$\theta \in \Theta := (1,\infty)$$, let $$f(x|\theta) := \frac{1}{\theta^x}$$ for each $$x \in \mathcal{X} := (0,\infty)$$. Then $$\int_{\Theta} f(x|\theta) \,d\theta := \begin{cases} \frac{1}{x-1} & \text{ if } x > 1 \\ \infty & \text{ if } 0 < x \leq 1 \end{cases}$$

(Would $$f(x|\theta) = 1/\theta^x$$ ever be used in practice?) Are there any other such examples? I am especially interested in examples such as last one, where the set $$\{x \in \mathcal{X}: \int_{\Theta} f(x|\theta) \,d\theta = + \infty \}$$ has positive and finite measure.

• Just to clarify: are you asking for which type of likelihood function $p(x|\theta)$ an improper prior (i.e., $\int_{\Theta} p(\theta) d\theta = \infty$) will result in an improper posterior (i.e., $\int_{\Theta} p(\theta|x) d\theta = \infty$)? As mentioned here, this is the case when the likelihood is flat. Apr 9, 2023 at 23:13
• The question is problematic in that the answer depends on the parameterisation chosen for the model $f(\cdot|\theta)$, since there is no preferential parameterisation for which the Uniform prior would be more justified than for others... Apr 10, 2023 at 8:01
• @Durden: Thanks for the link, I read through the thread. When you say "when the likelihood is flat", do you mean something like $f(x|\theta) = \theta x^{\theta - 1}, \, 0 \leq x \leq 1, \, \theta > 0$ in the case where $x = 0$ ? Do you know of any other non-pathological examples that one might actually encounter in practice? Apr 10, 2023 at 13:39
• @Leonidas the likelihood being 'flat' generally hints at identification issues (here's one example from the archives; I'm sure there are others): the likelihood as constructed considers a wide range of parameter values as (nearly or perfectly) equally likely, and as a result your posterior will just be your prior. This problem has many names across disciplines, but you'll find the most active recent discussion under the keyword 'posterior collapse' in ML. Apr 10, 2023 at 17:47