I just learned about improper priors in the Bayesian framework and was wondering if there could be any use in working with ''improper'' likelihoods. That is, if we believe there is something like overdispersion in the observed data relative to the models under considerations, could we propagate this uncertainty through the likelihood? If so, what might this look like, and is there any relation to quasi-likelihood in GLMs?


It depends on what you call an "improper likelihood". A general definition could be related to M-estimators, which "are a broad class of estimators, which are obtained as the minima of sums of functions of the data". These functions are not necessarily (log-)density or integrable functions. Thus, in that sense, M-estimators can be seen as estimators coming from an "improper likelihood", in a very vague sense. I emphasize that likelihood functions are defined in terms of probability models. If you have uncertainty about the model, you can use a more flexible model, or even a Bayesian nonparametric model.

However, their genesis is entirely different: M-estimators are developed in order to produce consistent estimators, while improper priors usually arise in the context of "objective Bayes" (noninformative priors).

Warning: if you use improper priors, the posterior distribution might also be improper (this requires a case by case analysis). If you combine an "improper likelihood" (in the sense that it does not arise from a probability model), with a prior (either proper or improper) this may also induce an improper posterior.

  • 2
    $\begingroup$ The connection with M estimation is interesting. I suppose improper likelihood would be a reasonable way to describe the approach. $\endgroup$ – Michael R. Chernick Apr 27 '17 at 14:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.