# Random likelihood function during maximizing process

Mostly common definition of Maximum Likelihood Estimator in certain parametric problem $$(\mathcal{X},\mathcal{B}(\mathbb{R}^n),\mathcal{P}=\{\mathbb{P}_\theta\mid\theta\in\Theta\})$$is what follows: $$\hat\theta_{MLE}(\mathbf{x})\equiv\underset{\theta\in\Theta}{\operatorname{argmax}}L_n(\theta\mid\mathbf{x}),\quad\forall\mathbf{x}\in\mathcal{X}$$ It means that for all n-dimensional data points $$\mathbf{x}$$ we have unique solution, which maximize Likelihood function. Then, during the process of maximizing Likelihood is considered as non-random function of $$\theta$$, with data given. I am interested under what assumption we can treat the Likelihood as random in process of deriving estimator- i look for more formal, measure-theoretic approach to this problem. I think that the likelihood should be measurable function, differentiable in $$\theta\quad a.s\quad\mathcal{P}$$. Am I right? What else?

• There is no need for the likelihood to be either measurable nor differentiable in $\theta.$ Insisting on that would introduce unnecessary restrictions and rule out the applicability of this theory in some significant situations.
– whuber
Mar 13, 2020 at 16:37
• The random function $L_n(\theta\mid\mathbf{x})$ should be measurable in $\theta$ (at the very least, to talk about existence of argmax you need more structure) $\mathcal{P}$-a.s. The argmax $\hat\theta_{MLE}$ should be a measurable function of data $\mathbf{x}$, as any estimator should be. There is no requirement that $L_n(\theta\mid\mathbf{x})$ be jointly measurable in $\theta$ and $\mathbf{x}$. Mar 13, 2020 at 20:59

Let $$\left(\mathcal X, \mathcal A, \left(\mathbb P_\vartheta\right)_{\vartheta \in \Theta}\right)$$ be a statistical model consisting of
• a set $$\mathcal X$$ (the sample space),
• a $$\sigma$$-algebra $$\mathcal A$$ on $$\mathcal X$$,
• a family of probability measures $$\left(\mathbb P_\vartheta\right)_{\vartheta \in \Theta}$$ on $$\mathcal A$$ with
• index set (the parameter space) $$\Theta$$ of cardinality bigger than one.
If, for all $$\vartheta \in \Theta$$, $$\mathbb P_\vartheta$$ is absolutely continuous w.r.t. a $$\sigma$$-finite measure $$\mu$$ on $$\mathcal A$$, then the likelihood function $$\mathcal L$$ is defined as the Radon–Nikodym derivative of $$\mathbb P_\vartheta$$ w.r.t. $$\mu$$: $$\mathcal L(\vartheta, x) \mathrel{:=} \frac{\mathrm d \mathbb P_\vartheta}{\mathrm d \mu}(x), \; \forall \, \vartheta \in \Theta, x \in \mathcal X,$$ and often viewed as a function in $$\vartheta$$.