# Use of Relative Likelihoods in Statistics?

In this article, the author lists several different methods (e.g. Wald, Wilson, Clopper-Pearson, etc.) that can be used to calculate Confidence Intervals for a Binomial Proportion, and then compares the advantages and disadvantages of these different methods.

One of the approaches that is used to compare the different Confidence Intervals is to use the notion of Relative Likelihoods. As I understand, a Relative Likelihood is a ratio between the actual likelihood function and the likelihood function based on the parameter estimates. The Relative Likelihood Function is the same in all 4 plots. And in each one of these plots, the horizontal line represents the Confidence Interval calculated using each of these 4 different methods.

I am having difficulty understanding the following point:

• Apparently, these plots can be used to show that some of these Confidence Intervals contain "implausible values".
• That is, in instances where the horizontal line extends past the boundaries of the Relative Likelihood Function - values beyond these points are considered "implausible"

I am trying to understand why exactly this is true - can someone please help me understand why are values of the Confidence Interval in regions outside of the Relative Likelihood Function considered as "implausible"?

Thanks!

References:

• I'd guess that the definition is informal and means that, also informally, some confidence intervals "stick out" above the relative likelihood in the plots. Those subsets of the CI correspond to parameter values $\theta$ such that $L(\theta)$ is less than the cutoff used to construct the likelihood interval (cutoff = 15%?) A followup question may be whether a (frequentist) confidence interval should care about this fact as long as it is well calibrated. Jan 3 at 10:21
• @dipetkov - I agree with you... perhaps you should expand that into an answer? Jan 4 at 1:24
• @jbowman There are related threads that mostly answer this question. I suggest the OP starts their research with How to create an interval estimate based on being within some percentage of the maximum value of the likelihood function?. If they want to read more about constructing confidence intervals using the long-run frequency vs using the likelihood, I recommend Chapter 2 of "In All Likelihood" by Y. Pawitan. PS: I don't think the term "plausible interval" is used much & arguing whether this term makes sense or not isn't very interesting, to me at least. Jan 4 at 2:09
• @dipetkov These look like some promising leads. Please do raise a flag if you find an exact duplicate of this question, or questions which, together, provide a complete answer to this question.
– Sycorax
Jan 4 at 16:20
• @dipetkov Understood! Writing an answer is the correct action, when you don't believe any existing questions are exact duplicates.
– Sycorax
Jan 4 at 17:37

Apparently, these plots can be used to show that some of these Confidence Intervals contain "implausible values"

These 'implausible values' are considered to be values that are outside the 95% likelihood interval. In the image this occurs when the horizontal line, representing the interval, crosses the normalized likelihood function.

Below we draw the plot and add some extra highlights with red color for the parts that are outside the likelihood interval. I am trying to understand why exactly this is true - can someone please help me understand why are values of the Confidence Interval in regions outside of the Relative Likelihood Function considered as "implausible"?

This is not generally true. It is just an arbitrary definition to set the likelihood interval equal to the boundary of plausible and implausible values.

In the article they made a table where the relative likelihood appears to be the best interval. It is on average the smallest interval and it has an average coverage of 95%. But this situation only occurs because of the (arbitrary) way that the average is computed. The true value of $$p$$ is considered as uniform distributed.

This is similar to the credible interval being optimal (Are there any examples where Bayesian credible intervals are obviously inferior to frequentist confidence intervals), and that is only true if the (assumed) prior is true. The likelihood interval is effectively equivalent to a credible interval with uniform prior for the parameter $$p$$. Then, if you consider the average based on a uniform distribution of $$p$$, you will get that the likelihood interval performs the best.

So the "good performance" is all based on using the uniform distribution of $$p$$ to compute the average.

Sidenote: the article is not very clear and consistent with their definition of the 'likelihood interval'. In the text they define it as:

The set of $$p$$ values for which $$R(p) \geq c$$ is called a $$100c\%$$ likelihood interval (LI).

This is the same as the definition used by Wikipedia. But, the percentage does not relate to confidence intervals. A smaller percentage $$100c\%$$ means a larger interval, and that is opposite to what happens with confidence intervals.

The region that they compute, $$[0.7658, 0.9965]$$, in the image/example is not a 95% likelihood region. Instead it is a 14.65% likelihood region, which is asymptotically equivalent to a 95% confidence region (based on the likelihood statistic asymptotically approaching a $$\chi^2$$ distribution according to Wilks' theorem).

The interval that I computed for the image, the bounds $$[0.7608, 0.9968]$$ are for a $$95\%$$ highest density region, using the likelihood function as the density.

• Thank you! Why is the uniform distribution of "p-values" important? I found a video on this topic: youtube.com/watch?v=UPQtjahe3j4 Jan 7 at 5:52
• @stats_noob the table in that article is generated by simulations. They don't give their method but it seems that the values of $p$ are randomly drawn or chosen with a uniform distribution. This is important because that choice of uniform values makes the likelihood interval equivalent to a credible interval whose performance is always better if the prior distribution is known. Which interval is better depends on the circumstances and in the article, circumstances where created that made the likelihood interval perform better. Jan 7 at 9:19
• The YouTube video is something completely different. It discusses p-values. In my answer it is about the parameter $p$ of a binomial distribution. Jan 7 at 9:22

The relative, or normalized, likelihood (function) $$\tilde{\mathcal{L}}$$ is defined as $$\mathop{\tilde{\mathcal{L}}}\left(\theta\right) = \frac{\mathop{\mathcal L}\left(\theta\right)}{\mathop{\mathcal L}\left(\hat\theta_\text{ML}\right)},$$ where $$\mathcal L$$ is the likelihood function and $$\hat\theta_\text{ML} = \underset{\theta \,\in\, \Theta}{\mathrm{arg\;max}}\mathop{\mathcal L}\left(\theta\right)$$.
Clearly, $$\mathop{\tilde{\mathcal{L}}}\left(\theta\right) \in \left[0, \mathop{\tilde{\mathcal{L}}}\left(\hat\theta_\text{ML}\right)\right] = \left[0, 1\right]$$.

While there exist pure likelihood approaches based on thresholding $$\tilde{\mathcal{L}}$$ at certain cutoff points $$c$$ (e.g., $$\theta$$s are: very plausible if $$\mathop{\tilde{\mathcal{L}}}\left(\theta\right) \in \left(1/3, 1\right]$$, plausible if $$\mathop{\tilde{\mathcal{L}}}\left(\theta\right) \in \left(1/10, 1/3\right]$$, less plausible if $$\mathop{\tilde{\mathcal{L}}}\left(\theta\right) \in \left(1/100, 1/10\right]$$, etc.), deriving confidence intervals with (asympotically) guaranteed coverage probabilities requires calibration of the relative likelihood via probability.

To that end, consider the likelihood ratio statistic $$W$$ given by $$W = 2 \cdot \ln\left(\frac{\mathop{\mathcal L}\left(\hat\theta_\text{ML}\right)}{\mathop{\mathcal L}\left(\theta\right)}\right) = -2 \cdot \ln\left(\mathop{\tilde{\mathcal{L}}}\left(\theta\right)\right),$$
which, under the Fisher regularity conditions, is asymptotically $$\mathop{\chi^2}\left(1\right)$$-distributed for a scalar parameter $$\theta$$.
Hence, an asymptotic $$\gamma\cdot 100\%$$ confidence set (an interval if the likelihood function is not multimodal) for $$\theta$$ is given by $$\left\{\theta:\mathop{\tilde{\mathcal{L}}}\left(\theta\right) \geq c\right\},\\ c = \exp\left(-\frac{1}{2}\mathop{\chi^2_\gamma}\left(1\right)\right),$$ where $$\mathop{\chi^2_\gamma}$$ denotes the $$\gamma$$-quantile of the $$\mathop{\chi^2}\left(1\right)$$ distribution.

In Figure 5 all horizontal lines seem to be drawn at the same cutoff $$\mathop{\tilde{\mathcal{L}}}\left(p\right) = c$$ that determines the likelihood ratio confidence interval. I guess $$c = \exp\left(-\frac{1}{2}\mathop{\chi^2_{0.95}}\left(1\right)\right) \approx 0.1465$$, which corresponds to an asymptotic $$95\%$$ likelihood ratio confidence interval for $$p$$.
If so, the "plausibility judgment" would be entirely based on whether parameter values in the confidence intervals under consideration are contained in the likelihood ratio confidence interval or not. To me, this doesn't seem well-justified.

Reference

Held, L., & Sabanés Bové, D. (2020). Likelihood and Bayesian inference: With applications in biology and medicine (Second edition). Springer.

• +1 though your answer has a great deal with overlap with @utobi's answer here: stats.stackexchange.com/a/596631/237901. Which is I pointed it to the OP in the first place. I think we should encourage doing research by reading existing answers. It's fun! Jan 5 at 17:50
• And the argument for constructing likelihood-based confidence intervals is well justified. Here are some references @utobi gave me: > The CI described in my post IS the likelihood-based CI; see, e.g., Severini (2000) "Likelihood Methods in Statistics", Pace and Salvan (1997) "Principles of Statistical Inference", Azzalini (1996) "Statistical Inference Based on the likelihood", among many many others. The theory is also explained in Chapter 2 of All Likelihood. The only thing that's not well justified is introducing new terms for things we already had a term for. Jan 6 at 10:36
• @dipetkov Sure, but there are also other well-justified (asymptotic) confidence intervals with similar or superior coverage and (expected) interval length which would each lead to different "plausibility judgments" if the parameter values in the confidence intervals in the given paper would be compared to the parameter values contained in them. Also, in the given paper, I'm not really convinced that, the Wilson confidence interval (say for small to moderate sample sizes) is inferior to the likelihood ratio confidence interval. Jan 6 at 11:11
• Yes, I agree. The fact is the paper cited here is a poorly written paper, that's all. What I'm worried about is that the knowledge (about likelihood intervals) is now spread across multiple threads. This is a big pet peeve of mine. I need to keep reminding myself that if a question asker cared to do research they would have done it before asking the question. Jan 6 at 11:11

In some literature the term "plausible" for parameters is used synonymously with "high likelihood". The idea is in principle the same as the idea behind statistical tests: Parameters are implausible for the observed data if under the parameter these data are very unlikely, measured by the likelihood. This is the starting point for basing inference about parameters as exclusively as possible on the likelihood, as advocated by some authors, e.g., famously, A. W. F. Edwards in his book "Likelihood". There is also Birnbaum's proof of the Likelihood Principle, often interpreted as "all evidential information about the parameter from the data is in the likelihood function" (which has been challenged, I think convincingly, by Evans, Fraser, Mayo and others). In this spirit, if the "relative likelihood" of a parameter value is very low compared to the maximum likelihood, it means that the parameter value is far less plausible than the maximum likelihood estimator.

Obviously even if we subscribe to this view, there needs to be a somewhat arbitrary definition of a cutoff value between "plausible" and "implausible" if we want to use these terms in a binary manner. The idea behind the likelihood-based CI is apparently to say that a parameter value is "too implausible" if its likelihood is so low, compared to the ML estimator, that it doesn't belong to a CI defined based on large likelihood only (which then is relative to the chosen confidence level). Bringing this together with the meaning of CIs with level $$\beta=1-\alpha$$, it means that for repeated experiments, only with probability $$\alpha$$ (say 0.01) a true parameter value will be declared "implausible".

Note however that the very definition of CIs runs to some extent counter to the likelihood "philosophy", as it relies on the sampling characteristics of the CI computation method, which are not comprehensively covered by the likelihood function (opponents of frequentist inference, particularly Bayesians, have occasionally used this argument against CIs, whereas frequentists have used it against the Likelihood Principle).

Regarding the originally cited paper, claiming as an advantage of the likelihood method for computing a CI that it coincides with "plausibility" is somewhat circular as the likelihood method does this by definition as far as plausibility doesn't mean anything else than "high likelihood". It is, if you want, not an additional feature of the likelihood method beyond its definition. However, this feature still gives the likelihood method some intuitive appeal.

This is an extended comment, to complement the great answers by @ChristianHennig, @SextusEmpiricus and @statmerkur, with background material from related Cross Validated threads. The theory of confidence intervals and likelihood intervals is interesting and complex; reading expositions from multiple points of view might help to understand it better.

The article you cite uses the terms "plausible" / "implausible" to mean that some values for the parameter $$p$$ have "high" / "low" likelihood — relatively, when compared with other values for $$p$$. These are not commonly used terms, so I suggest you don't use them either.

Rather than:

The maximum likelihood estimate (MLE) $$p$$ of $$\hat{p}$$ is the most plausible value of $$p$$ in that it makes the observed sample most probable.

It's better to say: The maximum likelihood estimate $$\hat{p}$$ of the parameter $$p$$ is the value which maximizes the likelihood. Math notation is clear and precise:

\begin{align} \hat{p}_{\text{MLE}}:\arg\max_{p}L_x(p) \end{align} where $$L_x(p)$$ is the likelihood function.

Equivalently, you can say that the MLE maximizes the log-likelihood $$\log L_x(p)$$ or minimizes the negative log-likelihood $$-\log L_x(p)$$.

And now for the references.