2
$\begingroup$

MLE (maximum likelihood estimation) can be defined mathematically for discrete or continuous variables. But there is a technical specificity about variables being neither discrete nor continuous. Instead they are a mixture of discrete and continuous. More precisely:

Assume you have a model with a parameter $\theta$ and an observable variable $X$ whose distribution depends on $\theta$. You have $x_1,x_2...x_n$ an independent sample of real-life observations of $X$.

If $X$ is a discrete variable, the maximum likelihood estimator is:

$$\hat\theta=\text{argmax}_\theta \left(\displaystyle\prod_{i=1}^n P_\theta(X=x_i)\right)$$

If $X$ has a density $p_\theta$, then you just use the density instead:

$$\hat\theta=\text{argmax}_\theta \left(\displaystyle\prod_{i=1}^np_\theta(x_i)\right)$$

Sometimes, $X$ is essentially continuous but has one or several atoms: the mixture of a continuous distribution and a discrete distribution. A good example : ML estimate of exponential distribution (with censored data).

I want to find a way to define MLE mathematically when you have atoms for students or people with some mathematical background but not so much in statistics. Ideally :

  • not too much theoretical or abstract
  • rather general
  • not uselessly confusing

I struggle. Any idea?

$\endgroup$
  • 1
    $\begingroup$ Possible duplicate of Maximum Likelihood Estimation (MLE) in layman terms $\endgroup$ – Tim Jul 15 '17 at 20:56
  • $\begingroup$ My question is a focus on the mathematical definition of MLE with neither discrete nor continuous variables. I think the question you are referring to is more about an intuitive general explanation of MLE. I've rephrased a bit. $\endgroup$ – Benoit Sanchez Jul 15 '17 at 21:11
  • 1
    $\begingroup$ But the definition is the same no matter if your data is continuous or discrete! Probability mass functions is a special kind of density function, so there is no problem with mixed data. $\endgroup$ – Tim Jul 15 '17 at 21:16
  • $\begingroup$ Not a special case (en.wikipedia.org/wiki/Probability_mass_function). Unless you talk of Random-Nikodym derivative to the counting measure. I especially want to avoid these theoretical things. Anyway, somebody could explain this of idea of yours as an answer. This would not make my question a duplicate. $\endgroup$ – Benoit Sanchez Jul 15 '17 at 21:23
  • 1
    $\begingroup$ But if you want to discuss mixed data type then you can't run away from those theoretical considerations! this thread gives nice introduction to probability densities. You basically need to introduce probability densities and discuss discrete data as a special case for it. If probability density is "probability per foot", then with discrete data you have obvious units you calculate it for. $\endgroup$ – Tim Jul 15 '17 at 21:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.