I have found the term "asymptotic power of a statistical test" only related to the Kolmogorov-Smirnov test (to be precise: asyptotic power = 1). What does this term acctually mean? In my opinion it should be someting like this: "if the alternative hypothesis is true, than for every significance level alpha there exists a sample size $n$ that the selected test would reject the null hypothesis". Is "my" definition correct? According to "my definition" the majority of classical tests (t-tset, ...) should have the asymptotic power 1, not only KS test. Am I right? ;)
$\begingroup$
$\endgroup$
2
-
$\begingroup$ Kolmogorov test is for distributions hence object lying in infinite dimensional space... not obvious to get asymptotic power=1. t-test is about testing the mean of a real valued random variable? $\endgroup$– robin girardCommented Sep 8, 2010 at 13:10
-
$\begingroup$ I haven't claimed that it is obvious, I just said that I have found this statement. So the natural question for me would be: what about other statistical tests - no matter what they are designed for. $\endgroup$– LanCommented Sep 8, 2010 at 13:33
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
2
The definition above (a fixed alternative, sample size going to infinity) is more precisely related to the consistency (or not) of a hypothesis test. That is, a test is consistent against a fixed alternative if the power function approaches 1 at that alternative.
Asymptotic power is something different. As Joris remarked, with asymptotic power the alternatives $\theta_n$ are changing, are converging to the null value $\theta_0$ (on the order of $\sqrt n$, say) while the sample size marches to infinity.
Under some regularity conditions (for example, the test statistic has a monotone likelihood ratio, is asymptotically normal, has asymptotic variance $\tau$ continuous in $\theta$, yada yada yada) if $\sqrt n(\theta_n - \theta_0)$ goes to $\delta$ then the power function goes to $\Phi(\delta/\tau - z_\alpha)$, where $\Phi$ is the standard normal CDF. This last quantity is called the asymptotic power of just such a test.
See Lehmann's $\underline{\mbox{Elements of Large Sample Theory}}$ for discussion and worked out examples.
By the way, yes, the majority of classical tests are consistent.
-
$\begingroup$ I don't thing consistency can have any sense in a goodness of fit test (such as KS test). You said "with asymptotic power the alternatives are changing" do you have an idea of how they change (different way of changing them will lead to different conclusions)? "Under some regularity conditions" is it easy to meet these regularity conditions in the case of Kolmogorov Smirnov test ? Did you find something about that in Lehmann's book ? $\endgroup$ Commented Nov 4, 2010 at 12:33
-
$\begingroup$ 1) Sure, it can. Let H0: X ~ N(0,1), suppose X1,...Xn ~ N(1,1), and let n go to infinity. That's consistency for the alternative $N(1,1)$. The KS-test is nonparametric so I bet it would be lots of fun to show consistency for all possible alternatives. 2) Yes, they change on the order of $\sqrt n$, say. 3) I'm not quite clear on the question(s): are you wanting to show the asymptotic power of the KS-test is 1? The first step would be to find a definition of asymp power that makes sense for a nonparametric test. I don't happen to know of one, in Lehmann's book or elsewhere. $\endgroup$– user1108Commented Nov 4, 2010 at 13:30
$\begingroup$
$\endgroup$
1
As I understood it, the asymptotic power is the hypothetical power when the effect size goes to zero and the sample size to infinity. Basically it should be 0 or 1, indicating whether the test cannot or can distinguish an arbitrary small deviation from the null hypothesis when the sample size is sufficiently large.
answered Sep 8, 2010 at 12:54
-
$\begingroup$ Thank you, Joris. So "my definition" was just partially correct. ;) Do you probably know the asyptotic power of Friedman test (non-parametric repeated measures ANOVA)? $\endgroup$– LanCommented Sep 8, 2010 at 13:43
$\begingroup$
$\endgroup$
Yes, you are right. I would only replace "there exists a sample size $n$ that the selected test would reject the null hypothesis" with "for every $e>0$ there exists a sample size $n_0$ such that the probability to reject the null hypothesis is greater than $1-e$ for all $n>n_0$".
