I am trying to understand power calculation for the case of the two independent sample t-test (not assuming equal variances so I used Satterthwaite).
Here is a diagram that I found to help understand the process:
So I assumed that given the following about the two populations and given the sample sizes:
mu1<-5
mu2<-6
sd1<-3
sd2<-2
n1<-20
n2<-20
I could compute the critical value under the null relating to having 0.05 upper tail probability:
df<-(((sd1^2/n1)+(sd2^2/n2)^2)^2) / ( ((sd1^2/n1)^2)/(n1-1) + ((sd2^2/n2)^2)/(n2-1) )
CV<- qt(0.95,df) #equals 1.730018
and then calculate the alternative hypothesis (which for this case I learned is a "non central t distribution"). I calculated beta in the diagram above using the non central distribution and the critical value found above. Here is the full script in R:
#under alternative
mu1<-5
mu2<-6
sd1<-3
sd2<-2
n1<-20
n2<-20
#Under null
Sp<-sqrt(((n1-1)*sd1^2+(n2-1)*sd2^2)/(n1+n2-2))
df<-(((sd1^2/n1)+(sd2^2/n2)^2)^2) / ( ((sd1^2/n1)^2)/(n1-1) + ((sd2^2/n2)^2)/(n2-1) )
CV<- qt(0.95,df)
#under alternative
diff<-mu1-mu2
t<-(diff)/sqrt((sd1^2/n1)+ (sd2^2/n2))
ncp<-(diff/sqrt((sd1^2/n1)+(sd2^2/n2)))
#power
1-pt(t, df, ncp)
This gives a power value of 0.4935132.
Is this the correct approach? I find that if I use other power calculation software (like SAS, which I think I have set up equivalently to my problem below) I get another answer (from SAS it is 0.33).
SAS CODE:
proc power;
twosamplemeans test=diff_satt
meandiff = 1
groupstddevs = 3 | 2
groupweights = (1 1)
ntotal = 40
power = .
sides=1;
run;
Ultimately, I would like to get an understanding that would allow me to look at simulations for more complicated procedures.
EDIT: I found my error. should have been
1-pt(CV, df, ncp) NOT 1-pt(t, df, ncp)