# What can we say about the likelihood function, besides using it in maximum likelihood estimation?

I found that often in literature that likelihood values are often used to compare different estimation method for the same model. And I got the impression that is the only way likelihood values are used.

However, I wonder what else we can say about the likelihood function. For example, can we compare two totally different models' likelihood functions? What's the minimum of the likelihood (obviously not zero)? What's a good guess about the maximum without going through into iterative method of estimation? (Can we build up inequality equations and claim boundaries for the likelihood function?)

• A likelihood function shows the relative likelihoods suggested by the observations of different potential parameter values, and therefore needs to be nonnegative. But nothing changes in the inference if you multiply the likelihood function by something positive which is independent of the parameters of interest, so there are no bounds to the function apart from $0$ Dec 14, 2018 at 18:58

1. Can we compare two totally different models' likelihood functions?

Not exactly, but you can compare those models using their likelihood functions indirectly by using the Akaike information criterion (AIC), the Bayesian information criterion (BIC), or the Deviance information criterion (DIC).

2. What's the minimum of the likelihood (obviously not zero)?

The likelihood function associated to some models can be lower bounded for some data sets, but this is not true in general. The only general lower bound is zero. More generally, the likelihood surface often contains flat ridges when the associated Fisher information matrix is singular at the true value of the parameters (see this paper).

3. What's a good guess about the maximum without going through into iterative method of estimation?

There is no general rule: this is more of an optimisation problem than something related to the likelihood function.

4. Can we build up inequality equations and claim boundaries for the likelihood function

The exact likelihood is defined as a probability (see Chapter 9 of this book or Wikipedia), then this object is upper bounded by $1$. The problem is that people typically use the continuous approximation of this function (see the aforementioned reference) which is not upper bounded in general and it might have singularities.

Other interesting scenarios where the likelihood function is important:

• Hypothesis testing: Likelihood ratio test.

• Some information criteria, for example AIC, BIC, and DIC.

• Bayesian inference: the posterior distribution is proportional to $Likelihood\times Prior$.

• The profile likelihood: used to obtain interval inferences, such as confidence intervals, for the parameters of interest.

Likelihood is a probability density which happens to be defined on spaces from which we get 'observed' data.

If we do not know that parameters of this probability density, then we call this 'a likelihood function of the parameters'.

There is nothing else magical about it.

1. Can we compare two totally different models' likelihood functions?

Yes , as long as the data remains the same.

2. What's the minimum of the likelihood (obviously not zero)?

Likelihood takes non-negative values (including zeros).

3. What's a good guess about the maximum without going through into iterative method of estimation?

It could potentially be any positive number.

4. Can we build up inequality equations and claim boundaries for the likelihood function

Not sure I fully understand.

• Likelihood is a probability density? Could you supply either a citation to a source where this is stated (or proved) or edit your answer to include a proof? Jul 9, 2013 at 19:06
• What I actually meant : f(x|\theta) is a pdf when viewed as a function of x with \theta fixed, and is a likelihood function when viewed as a function of \theta with x fixed. For convenience i called the expression "f(x|\theta)" as likelihood, and the called the latter "likelihood function". I think I should edit it. Jul 9, 2013 at 21:19