3
$\begingroup$

Let's suppose we are trying to compare two hypotheses for a single parameter $\theta$. The null hypothesis $H_0$ is that $\theta = \theta_0$, and the alternative is that $\theta ≥ \theta_0$.

The Karlin-Rubin theorem, as I learned it, tells us that if there is such a statistic $T(X)$ which has the monotone likelihood ratio (MLR) property, meaning that the function

$$ \frac{P\left(T(X)|\theta_1\right)}{P\left(T(X)|\theta_0\right)} $$

is monotonically non-decreasing in $T$, that we can obtain a UMP test by thresholding samples based on the value of $T(X)$.

Question: what is the relationship between $T(X)$ being MLR and $T(X)$ being a sufficient statistic?

  • Is it required for the Karlin-Rubin theorem that $T(X)$ also be sufficient, or just MLR?
  • Are there any implications regarding being MLR and being sufficient?
    • Do we have MLR $\to$ sufficient and/or sufficient $\to$ MLR?
  • What about regarding being MLR and being minimal sufficient?
    • Do we have minimal sufficient $\to$ MLR and/or MLR $\to$ minimal sufficient?
  • Are these things related in any way at all?

In short, I see lots of posts on here talking about sufficient statistics in the setting of the Karlin-Rubin theorem, but as I'm reading about it, it seems like the MLR property is the criterion that really matters.

$\endgroup$
1
  • 2
    $\begingroup$ If $T$ were not sufficient for $\theta$ could you base the most powerful level-$\alpha$ test of any particular $\theta_0$ vs $\theta_1$ on $T$? $\endgroup$ Commented Jan 20 at 23:15

1 Answer 1

4
$\begingroup$

The Neyman–Pearson lemma states that the most powerful tests of $H_0: \theta=\theta_0$ vs $H_1: \theta=\theta_1$ are based on the likelihood ratio statistic for the full sample

$$ \frac{P\left(X|\theta_1\right)}{P\left(X|\theta_0\right)}>k $$

Only when $T$ is sufficient for $\theta$ can you equate $P(T(X)|\theta)$ to $P(X|\theta)$; & only when the likelihood ratio is a monotonic function of a scalar $T$ can you equate the test to $T(X)> k'$.

Sufficiency & monotone likelihood ratio are distinct distributional properties, which $T$ may enjoy neither or either or both of.

  • $T$'s being sufficient doesn't imply a monotone likelihood ratio in $T$. Consider a single observation $x$ from a Cauchy distribution with unknown location $\mu$: $x$ is sufficient for $\mu$, but the likelihood ratio for $\mu_1$ > $\mu_0$ increases from $x-\mu_0$ to a maximum & thereafter declines. See https://stats.stackexchange.com/a/635195/17230 for details.

  • A monotone likelihood ratio in $T$ doesn't imply $T$ is sufficient. Let $X_1, \ldots, X_n$ be i.i.d., with $S(X_1, \ldots, X_n)$ sufficient for the parameter $\theta$, & a monotone likelihood ratio in $S$. Now let $T=S(X_1)$: $T$ is clearly not sufficient, yet the likelihood ratio in $T$ is still monotone.

  • $T$ needn't be minimal sufficient. When the minimal sufficient statistic $M$ has a discrete distribution, a uniformly most powerful test may need to be based on $T(X) = M(X) + U$, where $U$ is a uniform random variable, to achieve a given significance level exactly (see e.g. How to design a randomized test?). But when $M$ has a continuous distribution, with no restrictions on its range, the requirement for $T$'s being a scalar entails minimal sufficiency. I'm not sure what can be said in general about in-between cases.

The point of the Karlin–Rubin theorem is just to state the conditions under which likelihood ratio statistics (for the full sample) order the sample space in the same way regardless of the particular parameter values stipulated in the alternative hypothesis (provided $\theta_1$ > $\theta_0$ of course), thus rendering all most powerful tests of the null equivalent.

$\endgroup$
4
  • $\begingroup$ I am rewriting my earlier comment to correct something. I get what you are saying for the Neyman-Pearson lemma. With two simple hypotheses looking at $\theta_0$ and $\theta_1$, the likelihood ratio itself is minimal sufficient wrt this restricted parameter space of two points. But I don't quite get how this relates to the Karlin-Rubin theorem. With the Karlin-Rubin theorem I thought we're looking for a statistic with this "MLR property." I don't get the relationship between this property and being sufficient. Is it that any MLR statistic is always sufficient? Minimal sufficient? $\endgroup$ Commented Jan 21 at 19:49
  • 2
    $\begingroup$ It's that the two unrelated properties, sufficiency of $T$ & a monotone likelihood ratio in $T$, together allow you to infer that the test based on $T$ is uniformly most powerful. I'll edit my answer to elaborate. $\endgroup$ Commented Jan 21 at 20:02
  • $\begingroup$ OK, thanks, that makes sense. And I think it would be minimal sufficient, not just sufficient, right? A non-minimal sufficient statistic would partition things less coarsely than the likelihood ratio would. $\endgroup$ Commented Jan 21 at 21:43
  • $\begingroup$ Thinking about it, do we perhaps have sufficient + MLR -> minimal sufficient? $\endgroup$ Commented Jan 21 at 21:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.