I am just wondering what kind of things could cause power to be lost in general (the opposite of reducing the noise of the data to increase the power of the test).
Could anyone give some summary or ideas how to think of the power issue?
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.Sign up to join this community
There is an enormous literature on this subject; I'll just give you a quick thumbnail sketch. Let's say you're testing groups' mean differences, as in T-tests. Power will be reduced if...
...variables are measured unreliably. This will in essence "fuzz over" the differences.
...variability within groups is high. This will render between-group differences less noteworthy.
...your criterion for statistical significance is strict, e.g., .001 as opposed to the more common .05.
...you are using a one-tailed test, hypothesizing that if there is a difference, a certain group will have the higher mean. This cuts down on opportunistically significant findings that occur with two-tailed tests.
...you are dealing with (or expecting) very slim mean differences in the first place. Small effect sizes decrease power.
If you google "power calculator" you'll find some nifty sites that demonstrate the way these factors interact, depending on values you input. And if you download the free Gpower3 program you'll have a nice tool for calculating power and considering the effects of different conditions--for a wide variety of statistical procedures.
Think of power as your ability to recognize the truthfulness of one of two competing data generating processes. It will obviously be easier when:
If the above do not hold, it will be harder to choose the "true" process. I.e., you will lose power.