Mostly common definition of Maximum Likelihood Estimator in certain parametric problem $(\mathcal{X},\mathcal{B}(\mathbb{R}^n),\mathcal{P}=\{\mathbb{P}_\theta\mid\theta\in\Theta\})$is what follows: $$ \hat\theta_{MLE}(\mathbf{x})\equiv\underset{\theta\in\Theta}{\operatorname{argmax}}L_n(\theta\mid\mathbf{x}),\quad\forall\mathbf{x}\in\mathcal{X} $$ It means that for all n-dimensional data points $\mathbf{x}$ we have unique solution, which maximize Likelihood function. Then, during the process of maximizing Likelihood is considered as non-random function of $\theta$, with data given. I am interested under what assumption we can treat the Likelihood as random in process of deriving estimator- i look for more formal, measure-theoretic approach to this problem. I think that the likelihood should be measurable function, differentiable in $\theta\quad a.s\quad\mathcal{P}$. Am I right? What else?

  • 1
    $\begingroup$ There is no need for the likelihood to be either measurable nor differentiable in $\theta.$ Insisting on that would introduce unnecessary restrictions and rule out the applicability of this theory in some significant situations. $\endgroup$
    – whuber
    Mar 13, 2020 at 16:37
  • $\begingroup$ The random function $L_n(\theta\mid\mathbf{x})$ should be measurable in $\theta$ (at the very least, to talk about existence of argmax you need more structure) $\mathcal{P}$-a.s. The argmax $\hat\theta_{MLE}$ should be a measurable function of data $\mathbf{x}$, as any estimator should be. There is no requirement that $L_n(\theta\mid\mathbf{x})$ be jointly measurable in $\theta$ and $\mathbf{x}$. $\endgroup$
    – Michael
    Mar 13, 2020 at 20:59

1 Answer 1


Let $\left(\mathcal X, \mathcal A, \left(\mathbb P_\vartheta\right)_{\vartheta \in \Theta}\right)$ be a statistical model consisting of

  • a set $\mathcal X$ (the sample space),
  • a $\sigma$-algebra $\mathcal A$ on $\mathcal X$,
  • a family of probability measures $\left(\mathbb P_\vartheta\right)_{\vartheta \in \Theta}$ on $\mathcal A$ with
  • index set (the parameter space) $\Theta$ of cardinality bigger than one.

If, for all $\vartheta \in \Theta$, $\mathbb P_\vartheta$ is absolutely continuous w.r.t. a $\sigma$-finite measure $\mu$ on $\mathcal A$, then the likelihood function $\mathcal L$ is defined as the Radon–Nikodym derivative of $\mathbb P_\vartheta$ w.r.t. $\mu$: $$ \mathcal L(\vartheta, x) \mathrel{:=} \frac{\mathrm d \mathbb P_\vartheta}{\mathrm d \mu}(x), \; \forall \, \vartheta \in \Theta, x \in \mathcal X, $$ and often viewed as a function in $\vartheta$.

Thus, the (random) likelihood process is the random process of the corresponding Radon–Nikodym derivative.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.