4
$\begingroup$

This is not homework! I have two very different answers for a power calculation I have done on the below data -

n = 50 in total, 25 in each group

hit = if the number called is then thrown on a die
probability of hit = 1/6
each person has 36 dice throws

control group = call random number then throw a die
wish group = wish for a number, call it out and then throw die

2 groups tested by 2 sample z-test because we know distribution parameters     
i.e mean chance is 6 (1/6 x 36 trials), standard dev is 2.23

control group hit total is 150
wish group hit total is 180

I know the result of the 2 sample z-test (z=-1.9025). That is not the question. Q.How do I calculate the power of the study (given I am interested in the difference between the groups rather than deviation from chance)?

I think my study is under powered but have done this calculation in the pwr package of R -

h0 <-6 #control group
ha <-7.2 #wish group
sigma <- sqrt(36*1/6*5/6)
d = (ha - h0)/sigma

pwr.norm.test(d = d, n = 25, sig.level = 0.05, alternative = "greater")

 Mean power calculation for normal distribution with known variance

          d = 0.5366563
          n = 25
  sig.level = 0.05
      power = 0.8504646
alternative = greater

I think this is wrong.

$\endgroup$
  • $\begingroup$ What is "the study" to which you refer? $\endgroup$ – whuber Apr 24 '14 at 21:58
  • 1
    $\begingroup$ The study is the experiment set out in the top box, i.e 50 people throwing a dice 36 times each and I am interested in what the power is of this set up to detect the difference observed between the groups $\endgroup$ – biggob1 Apr 24 '14 at 22:19
2
$\begingroup$

You are correct that the response you are getting from the r function is wrong. The R function pwr.norm.test is calculating the power of a 1 sample z test for the given sample size, effect size, and significance level. But you need the power for a 2 sample z test.

I do not think that you will easily be able to find a function in R to perform the desired power analysis because under the alternative, not only is your mean different, but the standard deviation for the wish group is different as well. Sigma for the wish group is $\sqrt{36*1/5*4/5} = 2.4$. Thus, under the alternative, the distribution of the difference in sample means is

$\bar{x}_2 -\bar{x}_1 \sim N( 1.2, 2.24^2/25 + 2.4^2/25) = N( 1.2, .43) $

Thus the power is the probability, $P_{H_a}(\frac{\bar{x}_2 -\bar{x}_1}{\sqrt{2.24^2/25 + 2.24^2/25}} > 1.645 ) = P_{H_a}(\bar{x}_2 -\bar{x}_1 > 1.04) $

From here, it should be a straightforward normal probability calculation to calculate the power using the distribution under the alternative.

$\endgroup$
  • 1
    $\begingroup$ Interesting but the standard deviation of both groups is the same sqrt(36 * 1/6 * 5/6). Please change your answer as this sounds useful $\endgroup$ – biggob1 Apr 25 '14 at 6:36
  • $\begingroup$ If the probability of guessing the correct number is no longer 1/6 under the alternative, then the sd changes accordingly. For each person in the wish group, the number of times they guess correctly is a binomial random variable with n=36 and p=1/5. The sd is thus sqrt( 36*1/5*4/5). $\endgroup$ – jsk Apr 25 '14 at 17:15
0
$\begingroup$

I've just downloaded a trial version of PASS which allowed me to do the calculation (power of 2 sample z-test) for which I got -

Power    N1  N2  N    δ    σ   Alpha
0.59970  25  25  50  1.2  2.2  0.050

This answers my question.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.