Skip to main content
fixed a typo
Source Link
Ute
  • 2.6k
  • 1
  • 8
  • 22

Notation becomes a bit less cluttered, if we write $\lambda := \lambda_1$ and $\rho := \lambda_2/\lambda_1$, that is, $\lambda_2=\rho\lambda$. Since the term $\log(x_1!x_2!)$ does not change the maximizer, we will then find the values $\rho, \lambda)$ that maximize $$ h(\lambda, \rho) = - \lambda(1+\rho) + x_1\log \lambda + x_2\log(\lambda\rho) $$ We find the root of $$ \frac{\partial h(\lambda,\rho)}{\partial\lambda}=-(1+\rho)+\frac{x_1+x_2}{\lambda}=0 $$ as $$\hat\lambda = \frac{x_1+x_2}{1+\rho},$$ and double checking the second partial derivative of $h$, we convince ourselves that this is actually a maximizer. Plugging it in to $h$, we get $$ h(\hat\lambda,\rho)=x_2\log\rho - (x_1+x_2)\log(1+\rho)-(x_1+x_2)+(x_1+x_2)\log(x_1+x_2), $$ and from there $$ \frac{\partial h(\hat\lambda,\rho)}{\partial \rho}=\frac{x_2}{\rho} -\frac{x_1+x_2}{1+\rho}=\frac{x_2-\rho x_1}{\rho(1+\rho)}. $$ The function $\rho\to h(\hat\lambda,\rho)$ takes one local maximum in $\rho = x_2/x_1$. $$\hat\rho = x_2/x_1.$$

  • If $x_1>x_2$, then $\hat\rho = x_2/x_1$ is the maximizer also under the constraint, and we get $$\hat\lambda_1 = x_1\quad \text{and}\quad \lambda_2 = x_2.$$$$\hat\lambda_1 = x_1\quad \text{and}\quad \hat\lambda_2 = x_2.$$
  • If $x_1\leq x_2$, then the local maximizer $x_2/x_1 \geq 1$. As $h$ is increasing in $\rho$ for $\rho\leq 1 \leq x_2/x_1$, choose $\hat\rho$ as large as allowed, so here: $\hat\rho = 1$, which gives $$ \hat\lambda_1 = \hat\lambda_2 = (x_1 + x_2)/2. $$

Notation becomes a bit less cluttered, if we write $\lambda := \lambda_1$ and $\rho := \lambda_2/\lambda_1$, that is, $\lambda_2=\rho\lambda$. Since the term $\log(x_1!x_2!)$ does not change the maximizer, we will then find the values $\rho, \lambda)$ that maximize $$ h(\lambda, \rho) = - \lambda(1+\rho) + x_1\log \lambda + x_2\log(\lambda\rho) $$ We find the root of $$ \frac{\partial h(\lambda,\rho)}{\partial\lambda}=-(1+\rho)+\frac{x_1+x_2}{\lambda}=0 $$ as $$\hat\lambda = \frac{x_1+x_2}{1+\rho},$$ and double checking the second partial derivative of $h$, we convince ourselves that this is actually a maximizer. Plugging it in to $h$, we get $$ h(\hat\lambda,\rho)=x_2\log\rho - (x_1+x_2)\log(1+\rho)-(x_1+x_2)+(x_1+x_2)\log(x_1+x_2), $$ and from there $$ \frac{\partial h(\hat\lambda,\rho)}{\partial \rho}=\frac{x_2}{\rho} -\frac{x_1+x_2}{1+\rho}=\frac{x_2-\rho x_1}{\rho(1+\rho)}. $$ The function $\rho\to h(\hat\lambda,\rho)$ takes one local maximum in $\rho = x_2/x_1$.

  • If $x_1>x_2$, then $\hat\rho = x_2/x_1$ is the maximizer also under the constraint, and we get $$\hat\lambda_1 = x_1\quad \text{and}\quad \lambda_2 = x_2.$$
  • If $x_1\leq x_2$, then the local maximizer $x_2/x_1 \geq 1$. As $h$ is increasing in $\rho$ for $\rho\leq 1 \leq x_2/x_1$, choose $\hat\rho$ as large as allowed, so here: $\hat\rho = 1$, which gives $$ \hat\lambda_1 = \hat\lambda_2 = (x_1 + x_2)/2. $$

Notation becomes a bit less cluttered, if we write $\lambda := \lambda_1$ and $\rho := \lambda_2/\lambda_1$, that is, $\lambda_2=\rho\lambda$. Since the term $\log(x_1!x_2!)$ does not change the maximizer, we will then find the values $\rho, \lambda)$ that maximize $$ h(\lambda, \rho) = - \lambda(1+\rho) + x_1\log \lambda + x_2\log(\lambda\rho) $$ We find the root of $$ \frac{\partial h(\lambda,\rho)}{\partial\lambda}=-(1+\rho)+\frac{x_1+x_2}{\lambda}=0 $$ as $$\hat\lambda = \frac{x_1+x_2}{1+\rho},$$ and double checking the second partial derivative of $h$, we convince ourselves that this is actually a maximizer. Plugging it in to $h$, we get $$ h(\hat\lambda,\rho)=x_2\log\rho - (x_1+x_2)\log(1+\rho)-(x_1+x_2)+(x_1+x_2)\log(x_1+x_2), $$ and from there $$ \frac{\partial h(\hat\lambda,\rho)}{\partial \rho}=\frac{x_2}{\rho} -\frac{x_1+x_2}{1+\rho}=\frac{x_2-\rho x_1}{\rho(1+\rho)}. $$ The function $\rho\to h(\hat\lambda,\rho)$ takes one local maximum in $$\hat\rho = x_2/x_1.$$

  • If $x_1>x_2$, then $\hat\rho = x_2/x_1$ is the maximizer also under the constraint, and we get $$\hat\lambda_1 = x_1\quad \text{and}\quad \hat\lambda_2 = x_2.$$
  • If $x_1\leq x_2$, then the local maximizer $x_2/x_1 \geq 1$. As $h$ is increasing in $\rho$ for $\rho\leq 1 \leq x_2/x_1$, choose $\hat\rho$ as large as allowed, so here: $\hat\rho = 1$, which gives $$ \hat\lambda_1 = \hat\lambda_2 = (x_1 + x_2)/2. $$
added 648 characters in body (R code for an illustration)
Source Link
Ute
  • 2.6k
  • 1
  • 8
  • 22

How do we find that maximizer?

How do we find that maximizer?
  • If $x_1>x_2$, then $\hat\rho = x_2/x_1$ is the maximizer also under the constraint, and we get $$\hat\lambda_1 = x_1\quad \text{and}\quad \lambda_2 = x_2.$$
  • If $x_1\leq x_2$, then the local maximizer $x_2/x_1 \geq 1$. As $h$ is increasing in $\rho$ for $\rho\leq 1 \leq x_2/x_1$, choose $\hat\rho$ as large as allowed, so here: $\hat\rho = 1$, which gives $$ \hat\lambda_1 = \hat\lambda_2 = (x_1 + x_2)/2. $$
Remarks on the Likelihood ratio itself
  • The alternative to $H_0:\lambda_1>\lambda_2$ is, strictly spoken $H_A: \lambda_1 \leq \lambda_2$. When you choose the complete parameter space instead, you would get likelihood ratio equal to 1 whenever $x_1\geq x_2$.

  • For one sided null hypotheses $\lambda_1 > \lambda_2$ or $\lambda_2 > \lambda_1$ and opposite alternative, nominator and denominator swap their place in the two cases $x_1>x_2$ and $x_1 < x_2$. When $x_1=x_2$, the likelihood ratio is 1.

  • You can easily adapt the calculations to nulhypoteses of the form $\lambda_1> 10 \lambda_2$ or $\lambda_1 < 1.5 \lambda_2$.

Code for the figure (R)
# countour plot of Poisson log likelihood

loglik <- function(lambda1, lambda2, x = c(3,5))
  dpois(x[1], lambda1, log = TRUE)+ dpois(x[2], lambda2, log = TRUE)

la1 <- seq(1, 7, 0.01)
la2 <- la1

ll <- outer(la1, la2, loglik)

# draw contour map of log likelihood function
contour(la1, la2, ll, method = "edge",
        xlab = expression(lambda[1]),
        ylab = expression(lambda[2]),
        main="log likelihood, x=(3,5)") 

# unconditional maximum likelihood estimate at x
points (3, 5, pch = 16)

# identity line separating Omega_0 and Omega_1
abline(0, 1)
# conditional maximum likelihood: local maximum at (4,4)
points(4, 4, pch = 10)

How do we find that maximizer?

  • If $x_1>x_2$, then $\hat\rho = x_2/x_1$ is the maximizer also under the constraint, and we get $$\hat\lambda_1 = x_1\quad \text{and}\quad \lambda_2 = x_2.$$
  • If $x_1\leq x_2$, then the local maximizer $x_2/x_1 \geq 1$. As $h$ is increasing in $\rho$ for $\rho\leq 1 \leq x_2/x_1$, choose $\hat\rho$ as large as allowed, so here: $\hat\rho = 1$, which gives $$ \hat\lambda_1 = \hat\lambda_2 = (x_1 + x_2)/2. $$
How do we find that maximizer?
  • If $x_1>x_2$, then $\hat\rho = x_2/x_1$ is the maximizer also under the constraint, and we get $$\hat\lambda_1 = x_1\quad \text{and}\quad \lambda_2 = x_2.$$
  • If $x_1\leq x_2$, then the local maximizer $x_2/x_1 \geq 1$. As $h$ is increasing in $\rho$ for $\rho\leq 1 \leq x_2/x_1$, choose $\hat\rho$ as large as allowed, so here: $\hat\rho = 1$, which gives $$ \hat\lambda_1 = \hat\lambda_2 = (x_1 + x_2)/2. $$
Remarks on the Likelihood ratio itself
  • The alternative to $H_0:\lambda_1>\lambda_2$ is, strictly spoken $H_A: \lambda_1 \leq \lambda_2$. When you choose the complete parameter space instead, you would get likelihood ratio equal to 1 whenever $x_1\geq x_2$.

  • For one sided null hypotheses $\lambda_1 > \lambda_2$ or $\lambda_2 > \lambda_1$ and opposite alternative, nominator and denominator swap their place in the two cases $x_1>x_2$ and $x_1 < x_2$. When $x_1=x_2$, the likelihood ratio is 1.

  • You can easily adapt the calculations to nulhypoteses of the form $\lambda_1> 10 \lambda_2$ or $\lambda_1 < 1.5 \lambda_2$.

Code for the figure (R)
# countour plot of Poisson log likelihood

loglik <- function(lambda1, lambda2, x = c(3,5))
  dpois(x[1], lambda1, log = TRUE)+ dpois(x[2], lambda2, log = TRUE)

la1 <- seq(1, 7, 0.01)
la2 <- la1

ll <- outer(la1, la2, loglik)

# draw contour map of log likelihood function
contour(la1, la2, ll, method = "edge",
        xlab = expression(lambda[1]),
        ylab = expression(lambda[2]),
        main="log likelihood, x=(3,5)") 

# unconditional maximum likelihood estimate at x
points (3, 5, pch = 16)

# identity line separating Omega_0 and Omega_1
abline(0, 1)
# conditional maximum likelihood: local maximum at (4,4)
points(4, 4, pch = 10)
Source Link
Ute
  • 2.6k
  • 1
  • 8
  • 22

As you said, it is difficult to find the maximum likelihood under the compound hypothesis $H:\lambda_1>\lambda_2$. Generally, finding the maximum is often not possible in the standard way for compound hypotheses that cover only a part of the parameter space.

It can sometimes help to reparametrize - also here, as you will see later. We have the likelihood $$ \mathcal{L}(\lambda_1, \lambda_2; x_1, x_2)= \frac{\mathrm{e}^{-\lambda_1} \lambda_1^{x_1}}{x_1!}\cdot \frac{\mathrm{e}^{-\lambda_2} \lambda_2^{x_2}}{x_2!} $$ but for convenience, we maximize the log likelihood instead, $$ \log\mathcal{L}(\lambda_1, \lambda_2; x_1, x_2)= -(\lambda_1+\lambda_2) + x_1\log\lambda_1 + x_2\log\lambda_2 - \log(x_1!x_2!). $$

By standard calculus, we find that this function is maximized over the unconstrainted parameter space, $(\lambda_1,\lambda_2)\in \mathbb{R}^+\times\mathbb{R}^+$, for $\hat{\lambda}_1=x_1$ and $\hat{\lambda}_2=x_2.$

The figure below shows a contour plot of the log likelihood for $x_1=3, x_2=5$. The black dot is the maximum likelihood estimator for the unconstrainted case. Under the null hypothesis, we require $\lambda_1>\lambda_2$: this is the region under the identity line. The maximizer for this region is depicted as a circle with a cross.

likelihood contour plot

How do we find that maximizer?

Notation becomes a bit less cluttered, if we write $\lambda := \lambda_1$ and $\rho := \lambda_2/\lambda_1$, that is, $\lambda_2=\rho\lambda$. Since the term $\log(x_1!x_2!)$ does not change the maximizer, we will then find the values $\rho, \lambda)$ that maximize $$ h(\lambda, \rho) = - \lambda(1+\rho) + x_1\log \lambda + x_2\log(\lambda\rho) $$ We find the root of $$ \frac{\partial h(\lambda,\rho)}{\partial\lambda}=-(1+\rho)+\frac{x_1+x_2}{\lambda}=0 $$ as $$\hat\lambda = \frac{x_1+x_2}{1+\rho},$$ and double checking the second partial derivative of $h$, we convince ourselves that this is actually a maximizer. Plugging it in to $h$, we get $$ h(\hat\lambda,\rho)=x_2\log\rho - (x_1+x_2)\log(1+\rho)-(x_1+x_2)+(x_1+x_2)\log(x_1+x_2), $$ and from there $$ \frac{\partial h(\hat\lambda,\rho)}{\partial \rho}=\frac{x_2}{\rho} -\frac{x_1+x_2}{1+\rho}=\frac{x_2-\rho x_1}{\rho(1+\rho)}. $$ The function $\rho\to h(\hat\lambda,\rho)$ takes one local maximum in $\rho = x_2/x_1$.

Under the constraint $\lambda_1 > \lambda_2$, only values $\rho < 1$ are admitted.

  • If $x_1>x_2$, then $\hat\rho = x_2/x_1$ is the maximizer also under the constraint, and we get $$\hat\lambda_1 = x_1\quad \text{and}\quad \lambda_2 = x_2.$$
  • If $x_1\leq x_2$, then the local maximizer $x_2/x_1 \geq 1$. As $h$ is increasing in $\rho$ for $\rho\leq 1 \leq x_2/x_1$, choose $\hat\rho$ as large as allowed, so here: $\hat\rho = 1$, which gives $$ \hat\lambda_1 = \hat\lambda_2 = (x_1 + x_2)/2. $$