1
$\begingroup$

By following the distribution I provided here on stackmathematics

Q. A biased die favours the number 2. It's rolled 4 times and 2 comes up twice. It's rolled again 4 times and 2 comes up once. Calculate the Likelihood and find the MLE.

The likelihood is: $L(\theta;3) = \binom{8}{3}\theta^3(1-\theta)^5$, whereas we have the local maximum at $\theta = \frac{3}{8}$.

I want to verify that, approximately $(0.0718, 0.6026)$ is a 2-likelihood region for $\theta$ by making a plot to indicate this.

From what I understand, the 2-likelihood region is given as the following:

The region $R_k = \{\theta \in \mathbb{R}: L(\theta;x) \ge e^{-k}L(\hat{\theta};x)\}$ for the k-unit likelihood region for parameters $\theta$ based on data $x$.

We know that $x=3$ and $\theta = \frac{3}{8}$, then we apply the following:

$$\binom{8}{3}\theta^3(1-\theta)^5 \ge e^{-2}L\left(\frac{3}{8};3\right)\approx0.035$$

So I think the next step is to plug $0.0718, 0.6026$ into $\theta$ to compare the inequality.

Something like:

$$\binom{8}{3}(0.0718)^3(1-0.0718)^5 \approx 0.014$$

However, $0.014 < 0.035$ and so $0.0718$ does not fall in the 2-likelihood region?

If this is the case, is there an easier way to check for values that fall within the confidence interval of the likelihood region on R?

$\endgroup$
2
  • $\begingroup$ Could you add a reference for that definition of a $k$-likelihood region? It's not a quite standard concept, I believe ... $\endgroup$ Mar 3, 2022 at 21:38
  • $\begingroup$ @kjetilbhalvorsen It was taken from my lecture notes - It's the only one that I know. However, I cannot pass these notes over unless I get permission from my university. Unfortunately, they do not give any reference to this. $\endgroup$ Mar 3, 2022 at 22:14

1 Answer 1

3
$\begingroup$

Let $a=3$ and $b=5,$ so that $\hat\theta = a/(a+b).$ Let the likelihood function be $L(\theta;a,b).$ Taking logarithms, express the region as

$$R_k = \{\theta\in[0,1]\mid \log L(\theta;a,b) - \log L(\hat\theta;a,b) + k \ge 0\}.$$

Because for $k \gt 0$ there will be a lower limit $l$ between $0$ and $\hat\theta$ and an upper limit $u$ between $\hat\theta$ and $1,$ use any decent root finder within each of these intervals. Here is a sketch of the situation, showing the graph of the difference in log likelihoods as a function of $\theta$ on the interval $[0,1].$

Figure

This is a general picture of how maximum likelihood confidence intervals are found in any one-parameter situation where the likelihood is smooth and has a unique global maximum in the interior of the parameter space. (Sometimes one or both limits don't exist, though, depending on how that parameter space is defined.)

For an interval with confidence $1-\alpha,$ $2k$ is the upper $1-\alpha$ quantile of the chi-squared distribution with one degree of freedom. For instance, for $95\%$ confidence $k= 3.84\ldots/2 \approx 1.92.$

For $k=2,$ the built in uniroot function in R computes $l=0.1060\ldots$ and $u=0.7157\ldots.$ The check of any putative limit, such as those mentioned in the question, is trivial: just compare it to these endpoints.

a <- 3
b <- 5
k <- 2
Lambda <- function(x) a * log(x) + b * log(1-x)
theta.hat <- a/(a+b)
f <- function(theta) Lambda(theta) - Lambda(theta.hat)
l <- uniroot(function(x) f(x) + k, c(0, theta.hat))$root
u <- uniroot(function(x) f(x) + k, c(theta.hat, 1))$root
(c(Lower=l, Upper=u))
    Lower     Upper 
0.1060292 0.7157523

Notice it was unnecessary to compute the constant factors $\binom{8}{3}$ in Lambda (the log likelihood function) because they cancel in taking the difference of log likelihoods. This, too, is a general phenomenon.

$\endgroup$
4
  • $\begingroup$ This is very different to my lecture notes. Why is your k-likelihood region different to mine - do you have any introductory material to cover your equation? Thanks for showing me the R-calculation, these are very new to me as I just touched on this course section. I have a question on the confidence interval; can we apply definitions of limit points to the k-likelihood region, and how useful are these to understand the region? $\endgroup$ Mar 3, 2022 at 22:43
  • $\begingroup$ I don't know how you obtained your limits, nor can I even guess, so I can't say why yours are different. The material I have outlined here is covered in most accounts of maximum likelihood estimation. $\endgroup$
    – whuber
    Mar 3, 2022 at 22:55
  • $\begingroup$ I have had a further look at my notes and compared it to your answer. You have shown me the generalised likelihood ratio test as per my lecture notes. The method I used does the following: "The values of $\theta$ within the k-unit likelihood region are those whose likelihood is at least within a factor $e^{-k}$ of the maximum". That being said, how can I derive the critical region given a test of $0.05$, as I have yet to properly understand this part. $\endgroup$ Mar 7, 2022 at 16:21
  • $\begingroup$ The second part of my post answers that question: one critical region comprises all likelihoods greater than a multiple of $e^{-k}$ of the maximum, where $k$ is (under certain conditions that commonly apply) computed as an upper percentile of a chi-squared distribution with a number of degrees of freedom equal to the number of parameters being estimated. There are other types of confidence interval procedures in use--this is just one of them. $\endgroup$
    – whuber
    Mar 7, 2022 at 17:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.