10
$\begingroup$

The power is a continuous function of the parameter. If a statistical test has a power of 0.8, does that mean the power function is 0.8 at all parameters except the null hypothesis? Or does it mean the power function is almost everywhere 0.8 or at most 0.8?

$\endgroup$
  • 1
    $\begingroup$ Power to do what? $\endgroup$ – Michael M Jan 23 at 20:15
  • $\begingroup$ Detect a difference that’s real $\endgroup$ – Numbers Jan 23 at 20:15
  • $\begingroup$ What kind of difference that is real? $\endgroup$ – Alexis Jan 23 at 21:32
  • 5
    $\begingroup$ The conventional* 80% is taken at a specific alternative -- e.g. for a location test, at a given difference in population mean (specified effect size). At smaller effect sizes power is smaller, at larger effect sizes, power is larger. $\qquad$ * well, conventional in some application areas. $\endgroup$ – Glen_b -Reinstate Monica Jan 23 at 23:38
36
$\begingroup$

You'll hear "this test has 80% power" as shorthand for a better statement like: "under a bunch of assumptions, including but not limited to this particular sample size and this particular true effect size, this test has an 80% probability of rejecting the null hypothesis with a two-sided alternative at a 5% significance level". Don't try to make sense of a statement like "this test has 80% power" unless there is a lot more detail provided. Any statement about power that is four words long is leaving out so much detail that it's effectively meaningless.

$\endgroup$
  • 3
    $\begingroup$ -1 "Statistics is complicated so don't try to make sense of it" is a very poor answer. $\endgroup$ – Alexis Jan 23 at 21:33
  • 6
    $\begingroup$ I like this answer. It addresses the implications in a statement about power (sample size, effect size, assumptions) while also establishing that, unless these implications are clear, a statement about power is insufficient. $\endgroup$ – Dave Jan 23 at 22:34
  • 23
    $\begingroup$ I think several of the comments under this answer are misinterpreting the point it's making. $\endgroup$ – Glen_b -Reinstate Monica Jan 24 at 6:37
  • 15
    $\begingroup$ @Alexis: "Statistics is complicated so don't try to make sense of it" is indeed a very poor answer. But it's also a poor description of this answer. A better description would be "Statistics has precise terms that have well-defined meanings, so if you want to be clear then use those terms in the correct manner." $\endgroup$ – Richter65 Jan 24 at 22:53
  • 4
    $\begingroup$ On this forum, which I genuinely appreciate and enjoy, I've found that in order for me to give a helpful answer, I need to discern the intent of the questioner. From the body of the question, not the title alone, it appeared that Numbers had, at very least, a basic understanding of power functions and was trying to figure out a cryptic statement that we've all heard many times before. So that is the question I tried to answer. $\endgroup$ – Robert Alan Greevy Jr PhD Jan 25 at 23:44
11
$\begingroup$

Statistical power is a function of statistical test, acceptable type I error rate, sample size, and effect size. It also depends on adherence to assumptions made about the data, such as assumptions of normality, independence of observations, etc, but often in a power analysis these are taken to be as presumed by the test used.

It seems like you are asking if the power is the same for all possible differences: no, the effect size has a strong influence on power for a given sample size.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.