4
$\begingroup$

Would it be valid to choose a probability distribution for assumptions based on the actual maximum likelihood of various distributions derived from MLE estimates?

For example, suppose I find MLE estimates of a normal distribution $[ \mu , \sigma^2 ] = [1,4]$, which I then substitute (along with $n$) back into the likelihood formula $L_N =(2 \pi \sigma^2 )^{-n/2} \exp(- \sum_{i=1}^n [(x_i - \mu)^2/(2 \sigma^2 )] )$ to obtain a number $\hat{L}_N$. Could I then go through the same process to obtain $\hat{L}_B$ for a beta distribution and directly choose $\max \{ \hat{L}_N , \hat{L}_B \}$ in deciding which of the two distributions would be more appropriate in further analysis?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
$\begingroup$

If you were to do that then you would be effectively just finding the MLE in a single problem where the distributional family you are using is the union of the family of normal distributions and the family of beta distributions. That is somewhat strange, since the class of distributions you form include two different parametric forms. Nevertheless, it is perfectly valid, insofar as what you obtain is still an MLE. Letting $\mathscr{N}$ denote the class of all normal distributions and $\mathscr{B}$ denote the class of all beta distributions, your MLE would be:

$$\hat{F} = \underset{F \in \mathscr{F}}{\text{arg max }} L_\mathbf{x}(F) \quad \quad \quad \mathscr{F} = \mathscr{N} \cup \mathscr{B}.$$

Taking $\hat{L}_N \equiv \underset{F \in \mathscr{N}}{\text{max }} L_\mathbf{x}(F)$ and $\hat{L}_B \equiv \underset{F \in \mathscr{B}}{\text{max }} L_\mathbf{x}(F)$ gives you:

$$L_\mathbf{x}(\hat{F}) = \underset{F \in \mathscr{F}}{\text{max }} L_\mathbf{x}(F) = \max (\hat{L}_N, \hat{L}_B).$$

Now, it is important to bear in mind that, although this is a valid MLE, you will need to establish whether or not the standard properties of the MLE hold in this case. Some standard properties of the MLE given in theorems pertaining to parametric models will hold and some will not. This depends on the required assumptions for these properties, which is complicated in this case. In determining whether a particular property applies, you will need to examine the proofs of these properties and the required assumptions, and you should bear in mind that in this case you no longer have a class of distributions where there is a smooth "path" between all distributions in the class (in a topological sense).$^\dagger$

You can certainly use this MLE as an estimator of the true distribution across a class composed of two distributional families. Whether this estimator has the desirable statistical properties that hold in simpler cases is a tricky question, and would require a detailed investigation of derivation of those properties in this case.

$^\dagger$ Actually, this part is a little tricky. It is made more complicated by the fact that the beta distribution is asymptotically normal for a broad class of limiting cases, so there is arguably a smooth path between all distributions in the class, through the asymptotic limits. Whether this is sufficient to obtain the required properties for proofs of standard properties of the MLE is something that would require investigation.

Improve this answer
$\endgroup$
4
  • $\begingroup$ I'm glad you're vague about "standard properties and behaviors," because most of those of interest--especially concerning standard errors--will not apply. Bear in mind a standard assumption used to derive these properties is that none of the distributions is singular with respect to any other, which is decidedly not the case here, and another assumption typically is that the parameter space is connected, which also is not the case. $\endgroup$
    – whuber ♦
    Commented Feb 26, 2019 at 23:39
  • 1
    $\begingroup$ Okay, I might look into this more and try to flesh out which properties would and would not hold. My statement in my answer might be too broad. $\endgroup$
    – Ben
    Commented Feb 27, 2019 at 0:37
  • $\begingroup$ @whuber: actually, standard errors will be valid, but it's a bit of a slight of hand. That is, asymptotically, the correct model will almost certainly be chosen by this method, so asymptotically, it's equivalent to just starting with the correct model. And since standard errors are only asymptotically valid anyways, you very technically don't lose that standard property. Of course, since we don't actually live in asymtopia, we should be concerned about the small sample behavior of these methods...for which we often have little theory! $\endgroup$
    – Cliff AB
    Commented Feb 27, 2019 at 1:15
  • $\begingroup$ @Cliff That's right: you have articulated one important source of concern. The usual machinery of MLE provides inadequate information about which model is the more appropriate one: all it can do is tell you it selected one as "likelier" and, conditional on that selection, it can provide estimated standard errors. $\endgroup$
    – whuber ♦
    Commented Feb 27, 2019 at 12:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.