4
$\begingroup$

I use fitdistr in R to select which distribution fits my data best.

I tried cauchy, weibull, normal, and gamma.

The log-likelihood is $-329.8492$ for cauchy, $-277.4931$ for gamma, $-327.7622$ for normal, $-279.0352$ for weibull.

Which one is the best? The one with the largest value (i.e., gamma) or the one with the largest absolute value (i.e., Cauchy)?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
7
$\begingroup$

The one with the highest likelihood is the one with the best chance of producing your data --- as long as the likelihoods can be compared.

At the very least, this requires that no constants have been left out (you can normally leave out constants because you normally compare things with the same constants), and that they all have the same number of parameters (which the usual distributions by these names will do).

Indeed, you could take a Bayesian point of view and compute relative probabilities under some prior, like a uniform (though I don't know that I'd use a uniform prior across those models; it would be a very very rare situation where I'd equally seriously entertain all of those).

i.e. "Conditional on only between these distributions, the relative probability is mostly on gamma (82%) or Weibull (18%)" ... but many people seem to deny that such a comparison can be validly made.

Improve this answer
$\endgroup$
5
  • $\begingroup$ I'd appreciate some comments about the following. I've believed that absolute log-likelihoods are incomparable but that relative log-likelihoods can be compared. For example if the likelihood ratio test for 10 covariates having a total of 18 degrees of freedom is 1000 under distribution 1 and 1200 under distribution 2, distribution 2 seems to be pulling more information out of the covariates through a better fit. Any thoughts? $\endgroup$
    – Frank Harrell
    Commented Mar 16, 2014 at 16:10
  • $\begingroup$ @FrankHarrell I wouldn't say I was an expert on the following, so I'd welcome it if you would pick it apart. I think in a Bayesian context (which is why I raised it), one can make a direct probability argument $P(\text{model}_i|\text{data})=P(\text{data}|\text{model}_i).P(\text{model}_i)/P(\text{data})$. The denominator is an unknown constant but it's the same for all models, (call it $c_d$). As long as we take care not to do anything that would change the relative $P(\text{data}|\text{model}_i)/P(\text{data}|\text{model}_j)$, that argument would seem to follow through as I suggest ... (ctd) $\endgroup$
    – Glen_b
    Commented Mar 16, 2014 at 20:17
  • $\begingroup$ @FrankHarrell (ctd) ... above. However, in the case of just dealing with likelihood itself, I don't recall any especially solid arguments that it does work that way (which is why the qualification "as long as they can be compared"; I then give some obvious necessary conditions - the question is whether they're also sufficient conditions). Indeed I have several times here on CV called for just such arguments, because it's a question I'd like clearly resolved one way or the other. (Can one use AIC to compare a lognormal log-linear model with a gamma GLM with log-link, for example?) $\endgroup$
    – Glen_b
    Commented Mar 16, 2014 at 20:29
  • $\begingroup$ Perhaps the way forward is to consider Bayes factors and posterior probabilities. The posteriors you wrote above are absolute quantities in a sense, so I'm not clear that achieved log-likelihood can stand in for them. $\endgroup$
    – Frank Harrell
    Commented Mar 16, 2014 at 20:37
  • 1
    $\begingroup$ @Frank log-likelihood certainly can't. But if (as I mention in my answer) we're careful not to lose any constants (or slightly more generally - as I mention in comments - we are careful about keeping the ratios constant), then in fact the P(data|model_i) values can be computed. $\endgroup$
    – Glen_b
    Commented Mar 16, 2014 at 20:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.