What can we say about the likelihood function, besides using it in maximum likelihood estimation?

Question

I found that often in literature that likelihood values are often used to compare different estimation method for the same model. And I got the impression that is the only way likelihood values are used.

However, I wonder what else we can say about the likelihood function. For example, can we compare two totally different models' likelihood functions? What's the minimum of the likelihood (obviously not zero)? What's a good guess about the maximum without going through into iterative method of estimation? (Can we build up inequality equations and claim boundaries for the likelihood function?)

Answer 1

1. Can we compare two totally different models' likelihood functions?

   Not exactly, but you can compare those models using their likelihood functions indirectly by using the Akaike information criterion (AIC), the Bayesian information criterion (BIC), or the Deviance information criterion (DIC).

2. What's the minimum of the likelihood (obviously not zero)?

   The likelihood function associated to some models can be lower bounded for some data sets, but this is not true in general. The only general lower bound is zero. More generally, the likelihood surface often contains flat ridges when the associated Fisher information matrix is singular at the true value of the parameters (see this paper).

3. What's a good guess about the maximum without going through into iterative method of estimation?

   There is no general rule: this is more of an optimisation problem than something related to the likelihood function.

4. Can we build up inequality equations and claim boundaries for the likelihood function

   The exact likelihood is defined as a probability (see Chapter 9 of this book or Wikipedia), then this object is upper bounded by $1$. The problem is that people typically use the continuous approximation of this function (see the aforementioned reference) which is not upper bounded in general and it might have singularities.

   Other interesting scenarios where the likelihood function is important:

   • Hypothesis testing: Likelihood ratio test.

   • Some information criteria, for example AIC, BIC, and DIC.

   • Bayesian inference: the posterior distribution is proportional to $Likelihood\times Prior$.

   • The profile likelihood: used to obtain interval inferences, such as confidence intervals, for the parameters of interest.

Answer 2

Likelihood is a probability density which happens to be defined on spaces from which we get 'observed' data.

If we do not know that parameters of this probability density, then we call this 'a likelihood function of the parameters'.

There is nothing else magical about it.

1. Can we compare two totally different models' likelihood functions?

   Yes , as long as the data remains the same.

2. What's the minimum of the likelihood (obviously not zero)?

   Likelihood takes non-negative values (including zeros).

3. What's a good guess about the maximum without going through into iterative method of estimation?

   It could potentially be any positive number.

4. Can we build up inequality equations and claim boundaries for the likelihood function

   Not sure I fully understand.