The Karlin-Rubin Theorem states that for $H_0 \colon \theta \leq \theta_0$, $H_1 \colon \theta >\theta_0$, for distributions that have monotone likelihood property, test given by thresholding the sufficient statistics $T$ is the uniformly most powerful.

I understand that for each $\theta_0 < \theta_1$, the likelihood ratio test is the uniformly most powerful test for $H_0 \colon \theta = \theta_0$ and $H_1 \colon \theta = \theta_1$, which is equivalent to looking at sufficient statistics due to monotone likelihood property.

I also understand our power function will be increasing.

However, I do not see how these properties lead to the proof of Karlin–Rubin Theorem. I'm struggling to see where we use the monotone likelihood ratio property.


3 Answers 3

  • MLR property is required to express the test $\varphi$ in terms of the statistic $T(\mathbf x) $ in that for $\theta_0<\theta_1,$ $$\frac{f_{\theta_1}(\mathbf x) }{f_{\theta_0}(\mathbf x) } \gtreqless k\iff T(\mathbf x) \gtreqless t_0.$$

  • The first part of the proof shows the power function is nondecreasing. So, for all $\theta\leq\theta_0,$ $$\mathbb E_\theta\varphi\leq \mathbb E_{\theta_0}\varphi=\alpha.$$ Then the test due to NP lemma is MP at $\theta_1$ among all level $\alpha$ tests for testing $\mathcal H_0: \theta\leq\theta_0$ against $\mathcal H_1: \theta=\theta_1.$

  • Now the class of all tests $\varphi$ such that $\mathbb E[\varphi|\theta\leq\theta_0]\leq \alpha$ is a subclass of those satisfying $\mathbb E[\varphi|\theta=\theta_0]\leq \alpha;$ so the MP test of the latter class must be MP for the former subclass too. The test constructed by NP lemma is MP and is not depending on the value of $\theta_1.$ This means it is UMP level $\alpha$ test for testing $\mathcal H_0: \theta\leq \theta_0$ against $\mathcal H_1: \theta> \theta_0.$


Here is a proof of the Karlin-Rubin theorem following Lehmann's classical book. There, the proof omits some details which I try to provide in this posting.

Throughout, I assume that

  • The family of densities $\{f_\theta:\theta\in\mathbb{R}\}$ is identifiable ($f_\theta\not\equiv f_{\theta'}$ whenever $\theta\neq\theta'$.
  • For any $\theta<\theta'$, the $f_{\theta'}(\mathbf{x})/f_{\theta}(\mathbf{x})=r_{\theta,\theta'}(T(\mathbf{x}))$, where $t\mapsto r_{\theta,\theta'}(t)$ is monotone nondecreasing.

Fix $\theta_0$ and $0<\alpha<1$. Let $c_l$ and $c_u$ be the smallest and largest $(1-\alpha)$- quantiles of the statistic $T(X)$ with respect to $P_{\theta_0}$ and define the test statistic

\begin{align} \widetilde{\psi}(X)=\left\{ \begin{array}{lcr} 1 &\text{if} & T(X))>c_l\\ \gamma &\text{if} & T(X)=c_l\\ 0 &\text{if} & T(X)<c_l \end{array}\right. \end{align}

It may happen that either

  1. $c_l=c_u=c$, in which case $P_{\theta_0}(T>c)\leq \alpha\leq P_{\theta_0}(T\geq c)$,
  2. or $c_l<c_u$ in which case, $P_{\theta_0}(T>c_l)=\alpha=P_{\theta_0}(T \geq c_u)$ and $P_{\theta_0}(T>c_u)\leq\alpha$; hence, $f_{\theta_0}$ vanishes on $\{\mathbf{x}: c_l<T(\mathbf{x})<c_u\}$

Let $\theta_1>\theta_0$ and denote by $\psi(X)$ the Neymann-Pearson statistic \begin{align} \psi(X)=\left\{ \begin{array}{lcr} 1 &\text{if} & r_{\theta_0,\theta_1}(T(X))>k\\ \gamma &\text{if} & r_{\theta_0,\theta_1}(T(X))=k\\ 0 &\text{if} & r_{\theta_0,\theta_1}(T(X))<k \end{array}\right. \end{align} where $k$ is the $(1-\alpha)$-quantile of $Y(X)=r_{\theta_0,\theta_1}((T(X))$ with respect to $P_{\theta_0}$ and $\gamma$ is taken so that $$\alpha = E_{\theta_0}[\psi(X)]$$

Define \begin{align} t_*:=\inf\{t\in \tau: r_{\theta_0,\theta_1}(t)\geq k\}\qquad t^*:=\inf\{t\in \tau: r_{\theta_0,\theta_1}(t)> k\} \end{align} Notice that $r_{\theta_0,\theta_1}(t)=k$ for $t\in\tau\cap(t_*,t^*)$.

Case (1): $c:=c_l=c_u$. Claim: $t_*\leq c$. Suppose the contrary, that $c<t_*$. Then \begin{align} \alpha&\geq P_{\theta_0}[T(X)>c]\geq P_{\theta_0}[r_{\theta_0,\theta_1}(T(X))\geq k]\\ &\geq P_{\theta_0}[r_{\theta_0,\theta_1}(T(X))>k]+ \gamma P_{\theta_0}[r_{\theta_0,\theta_1}(T(X))= k]=\alpha \end{align} Hence $\alpha=P_{\theta_0}[T>c]$.

  • If $\{r_{\theta_0,\theta_1}(T(X))\geq k\}=\{T(X)>t_*\}$, \begin{align} \alpha&=P_{\theta_0}[T(X)>c]=P_{\theta_0}[c<T(X)\leq t_*] +P_{\theta_0}[T(X)>t_*]\\ &\geq P_{\theta_0}[r_{\theta_0,\theta_1}(T(X))>k] + P_{\theta_0}[r_{\theta_0,\theta_1}(T(X))=k]\\ &\geq P_{\theta_0}[r_{\theta_0,\theta_1}(T(X))>k] + \gamma P_{\theta_0}[r_{\theta_0,\theta_1}(T(X))=k]=\alpha \end{align} Hence $P_{\theta_0}[T>t_*]=\alpha$. Since $c$ is the (unique) $(1-\alpha)$-quantile of $T(X)$, then $t_*=c$, contradiction!
  • Similarly, if $\{r_{\theta_0,\theta_1}(T(X))\geq k\}=\{T(X)\geq t_*\}$, then \begin{align} \alpha&=P_{\theta_0}[T(X)>c]=P_{\theta_0}[c<T(X)< t_*] +P_{\theta_0}[T(X)\geq t_*]\\ &\geq P_{\theta_0}[r_{\theta_0,\theta_1}(T(X))>k] + P_{\theta_0}[r_{\theta_0,\theta_1}(T(X))=k]\\ &\geq P_{\theta_0}[r_{\theta_0,\theta_1}(T(X))>k] + \gamma P_{\theta_0}[r_{\theta_0,\theta_1}(T(X))=k]=\alpha \end{align} Hence $P_{\theta_0}[T(X)\geq t_*]=\alpha$. Since $c$ is the (unique) $(1-\alpha)$-quantile of $T(X)$, then $t_*=c$, contradiction!

This proves the claim and so, $t_*\leq c$. Now, from $$\mathbb{P}_{\theta_0}[T(X)>t_*]\leq P_{\theta_0}[r_{\theta_0,\theta_1}(T(X))>k]\leq\alpha,$$ and the fact that $c$ is the unique $(1-\alpha)$-quantile of $T(X)$ under $P_{\theta_0}$, it follows that $c\leq t^*$. Therefore, $$t_*\leq c\leq t^*$$

Case (2): $c_l<c_u$. It follows that $f_{\theta_0}$ vanishes in $T^{-1}(\tau\cap(c_1,c_u))$. Since $t\mapsto r_{\theta_0,\theta_1}(t)$ is monotone no decreasing, then either $\tau\cap(c_l,c_u)=\emptyset$ or $r_{\theta_0,\theta_1}$ takes the value $\infty$ on $(c_l,c_u)\cap\tau$, This means that $$t_*\leq c_l\leq t^*$$

We are now in position to conclude the proof of Karlin-Rubin's theorem. Observe that as $t_*\leq c_l\leq t^*$, we have that on $\{r_{\theta_0,\theta_1}(T(X))\neq k\}$, when $T(X)>c_l$, $r_{\theta_0,\theta_1}(T(X))>k$, and when $T(X)<c_l$, $r_{\theta_0,\theta_1}(T(X))<k$.

It follows from Neymann-Pearson's lemma that $\widehat{\psi}(X)$ is an UMP statistic of size $\alpha$ for testing $\bar{H_0}: \theta=\theta_0$ against $\bar{H_1}: \theta=\theta_1$.

  • Notice that that $\widetilde{\psi}(X)$ does not depend on the choice of $\theta_1$. Hence, $\widetilde{\psi}(X)$ is an UMP test for $\bar{H}_0:\theta=\theta_0$ against $H_1: \theta>\theta_0$.

  • The identifiablitly of the population $\{P_\theta:\theta\in\mathbb{R}\}$ implies that the power $$\beta(\theta_0)=E_{\theta_0}[\widetilde{\psi}(X)]<\beta(\theta_1)=E_{\theta_1}[\widetilde{\psi}(X)]$$ for all $\theta_1>\theta_0$.

  • The arguments used to prove the UMP property of $\widetilde{\psi}(X)$ for testing $\bar{H}_0:\theta=\theta_0$ against $\bar{H}_1: \theta=\theta_1$ ($\theta_0<\theta_1$) also show that $\widetilde{\psi}(X)$ is a UMP of size $\beta(\theta)=E_\theta[\widetilde{\psi}(X)]$ for testing $\theta$ against $\theta'$ for any $\theta<\theta'$.

  • In particular, $\widetilde{\psi}$ is the UMP test of size $\alpha$ for testing $H_0:\theta\leq \theta_0$ against $H_1:\theta>\theta_0$.


Let's take an example with the one sided t-test.

We observe a t-statistic $T$ and the null hypothesis is that $H_0: \mu = 0$ in which case the t-statistic is t-distributed. And the alternative hypothesis is that $H_0: \mu > 0$ in which case the t-statistic is non-central t-distributed.

Below is an image of those distributions and the likelihood ratios (we use $\nu =5$)


The monotone likelihood property means that the likelihood ratio function is either all decreasing or all increasing. This means that the region of maximum likelihood-ratio is for any of the alternative point hypotheses among the composite hypotheses described by an inequality of the same form e.g. "reject if $T_{\text{observed}}>T_c$". The value $T_c$ is eventually chosen based on the distribution under the null hypothesis and the same for every value $\mu_a$ of the alternative hypothesis value.

The monotone property makes that the rejection region will be a region dependent on a single inequality, and independent of the actual value of the true parameter $\theta$ within the range of possibilities of the composite hypothesis.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.