Example of a non-measurable maximum likelihood estimator If you have a measurable parameter space $(\Theta, \mathcal{F})$ and a parametric family of probability measures $(P_\theta)_{\theta \in \Theta}$ on a measurable space $(\mathcal{X}, \mathcal{B})$ that have densities $(f_\theta)_{\theta \in \Theta}$ with respect to some dominating measure $\mu$, then you might define the maximum likelihood estimator $\hat{\theta} : \mathcal{X} \to \Theta$ as
$$
\tag{1}\label{1}
\hat{\theta}(x)
= \operatorname*{arg\,max}_{\theta \in \Theta} f_\theta(x)
$$
for all $x \in \mathcal{X}$.
This is an abstract version of how MLEs are introduced in some statistics texts and classes, but there are a few issues.
For one thing, a global maximum might not be attained, or there might be multiple global maxima, so \eqref{1} isn't necessarily well-defined.
Moreover, if it happens to be well-defined, then is it measurable?
Certainly all examples of maximum likelihood estimators one sees in applied statistics courses (at least that I've seen) are measurable.
Of course you can get around these issues by defining MLEs in a different way: for example, you might call a function $\hat{\theta} : \mathcal{X} \to \Theta$ a maximum likelihood estimator if it is a priori measurable and there exists a set $N \in \mathcal{B}$ such that $\mu(N) = 0$ and
$$
\tag{2}\label{2}
f_{\hat{\theta}(x)}(x) = \sup_{\theta \in \Theta} f_\theta(x)
$$
for all $x \in \mathcal{X} \setminus N$.
However, oftentimes \eqref{2} isn't how maximum likelihood estimation is introduced, and I'm more interested in the question of measurability of functions $\hat{\theta}$ satisfying \eqref{1} or \eqref{2}.

Questions.
Are there any explicit examples in which some reasonable interpretation of \eqref{1} is well-defined (at least off a $\mu$-null set) but not measurable?
Alternatively, can a function $\hat{\theta}$ satisfying \eqref{2} (again, at least off a $\mu$-null set) exist when no such measurable function exists?
What about if $(\Theta, \mathcal{F})$ and $(\mathcal{X}, \mathcal{B})$ are standard Borel spaces?
If there isn't a constructive example, then is there a proof of the existence of an example?

There are various results that give criteria under which there cannot be examples of the kind I'm looking for, thereby restricting the class of spaces and densities that might possess such examples.
Example 1.
Lemma 2 in Jennrich (1969) implies that if


*

*$\Theta$ is a compact subset of a Euclidean space and $\mathcal{F}$ is its Borel $\sigma$-algebra, and

*the function $\theta \mapsto f_\theta(x)$ from $\Theta$ into $\mathbb{R}$ is continuous for every $x \in \mathcal{X}$,


then there exists a measurable function $\hat{\theta} : \mathcal{X} \to \Theta$ such that \eqref{2} holds for all $x \in \mathcal{X}$.
Example 2.
Theorem 3.1 in Wagner (1980) (the Kuratowski and Ryll-Nardzewski selection theorem) implies that if


*

*$(\Theta, \mathcal{F})$ is a Polish space with its Borel $\sigma$-algebra,

*$(\mathcal{X}, \mathcal{B})$ is any measurable space,

*for each $x \in \mathcal{X}$, the set
$$
  F(x)
= \big\{\theta \in \Theta : \text{$f_\theta(x) \geq f_\psi(x)$ for all $\psi\in\Theta$}\big\}
  $$
is nonempty and closed, and

*for every open $U \subseteq \Theta$, the set
$$
  \big\{x \in \mathcal{X} : F(x) \cap U \neq \emptyset\big\}
  $$
belongs to $\mathcal{B}$,


then there exists a measurable function $\hat{\theta} : \mathcal{X} \to \Theta$ such that $\hat{\theta}(x) \in F(x)$ for all $x \in \mathcal{X}$ (i.e., \eqref{2} holds for all $x \in \mathcal{X}$).
Thus, under the additional assumptions of Example 1 or Example 2 there always exists a measurable maximum likelihood estimator.
Example 3.
Theorem 7.50(b) in Bertsekas and Shreve (1996) implies that if


*

*$(\Theta, \mathcal{F})$ and $(\mathcal{X}, \mathcal{B})$ are standard Borel spaces, and

*for every $c \in \mathbb{R}$, the set
$$
\big\{ (x, \theta) \in \mathcal{X} \times \Theta : f_\theta(x) > c \big\}
$$
is an analytic subset of $\mathcal{X} \times \Theta$,


then the set
$$
I = \left\{ x \in \mathcal{X} : \text{$f_\theta(x) = \sup_{\psi\in\Theta} f_\psi(x)$ for some $\theta \in \Theta$}\right\}
$$
is universally measurable. Moreover, for each $\varepsilon > 0$ there exists a universally measurable function $\hat{\theta} : \mathcal{X} \to \Theta$ such that \eqref{2} holds for all $x \in I$, and for all $x \in \mathcal{X} \setminus I$,
$$
f_{\hat{\theta}(x)}(x)
\geq \begin{cases}
\sup_{\psi \in \Theta} f_\psi(x) - \varepsilon, & \text{if $\sup_{\psi \in \Theta} f_\psi(x) < \infty$,} \\
\varepsilon^{-1}, & \text{if $\sup_{\psi \in \Theta} f_\psi(x) = \infty$.}
\end{cases}
$$
Example 3 doesn't necessarily guarantee the existence of a measurable maximum likelihood estimator (for instance, since there are universally measurable sets which are not Borel), but it is arguably in the same spirit as Example 2.
 A: Here is a contrived example.
Let $(\mathcal{X}, \mathcal{B})$ be the interval $[0, 2]$ with its Borel $\sigma$-algebra, let $(\Theta, \mathcal{F})$ be the interval $[1, 2]$ with its Borel $\sigma$-algebra, and let $P_\theta$ be the uniform distribution on $[0, \theta]$ for each $\theta \in \Theta$.
The family $(P_\theta)_{\theta \in \Theta}$ is dominated by the restriction of Lebesgue measure to $\mathcal{X}$ (call it $\mu$ as in the question), and a sensible choice of density for $P_\theta$ with respect to $\mu$ is
$$
f_\theta = \theta^{-1} \mathbf{1}_{[0, \theta]}.
$$
The maximum likelihood estimator $\hat{\theta} : \mathcal{X} \to \Theta$ is then given by
$$
\hat{\theta}(x) = \max\{1, x\},
$$
which is certainly measurable.
However, suppose I'm not sensible, and that I have a favorite Vitali set $V \subseteq [0, 1]$, and I define a family $(g_\theta)_{\theta \in \Theta}$ of functions $\mathcal{X} \to \mathbb{R}$ as follows:


*

*if $\theta - 1\in V$, then $g_\theta = 3 \mathbf{1}_{\{\theta - 1\}} + \theta^{-1} \mathbf{1}_{[0, \theta] \setminus \{\theta - 1\}}$;

*if $\theta - 1\notin V$, then $g_\theta = f_\theta$.


In other words,
$$
g_\theta(x)
= \begin{cases}
3, & \text{if $x \in V$ and $x = \theta - 1$,} \\
f_\theta(x), & \text{otherwise}
\end{cases}
$$
for all $x \in \mathcal{X}$ and $\theta \in \Theta$.
Since each $g_\theta$ is a modification of $f_\theta$ on at most a singleton, the family $(g_\theta)_{\theta \in \Theta}$ is another family of densities for $(P_\theta)_{\theta \in \Theta}$ with respect to $\mu$.
Now one can check that there is a unique function $\hat{\vartheta} : \mathcal{X} \to \Theta$ such that
$$
g_{\hat{\vartheta}(x)}(x) = \sup_{\theta \in \Theta} g_\theta(x)
$$
for each $x \in \mathcal{X}$; this function is given by
$$
\hat{\vartheta}(x) = \begin{cases}
\max\{1, x\}, & \text{if $x \notin V$,} \\
x + 1, & \text{if $x \in V$,}
\end{cases}
$$
and it is not measurable (for example, since $\{x \in \mathcal{X} : \hat{\vartheta}(x) = x + 1\} = \{0\} \cup V$).
Thus, in the setup with the family of densities $(g_\theta)_{\theta \in \Theta}$, there exists a function $\hat{\vartheta} : \mathcal{X} \to \Theta$ such that
$$
g_{\hat{\vartheta}(x)}(x) = \sup_{\theta \in \Theta} g_\theta(x)
$$
for all $x \in \mathcal{X}$, but no such measurable function exists.
This example merely shows that the the question of whether a measurable maximum likelihood estimator exists is sensitive to the choice of version of the Radon-Nikodym derivative $dP_\theta / d\mu$. When we used the much more reasonable family of densities $(f_\theta)_{\theta \in \Theta}$ we had no trouble obtaining a measurable maximum likelihood estimator, but when we artificially perturbed each of these densities on a null set to obtain $(g_\theta)_{\theta \in \Theta}$, we ran into problems.
This example is inspired by Example 2.2 in Pfanzagl (1969), wherein a similar but much more subtle modification of densities yields a scenario in which no maximum likelihood estimator exists at all, measurable or not.
