Sample size for CI precision

Question

I am interested in calculating the required sample size to achieve some precision using a CI for the mean.

Let's say that I wish to build a 95% CI for the mean, so that the difference between the population mean and the sample mean won't be larger than 2 units (with 95% confidence), and I assume a S.D of 5 units. To do so, I play a bit with the formula, extract N, and being an exact calculation, I get what I want. In the case of t (rather than z), the computer will do it and give me N = 27. Now I tried to do it with SAS, and I found this source.

Apparently, SAS adds another dimension to the calculation, which is an equivalent to the power of hypothesis testing. I don't understand why is it necessary in the first place. Using the CI formula, if N = 27 give a precision of 2, isn't it enough ? For 80% "power" I get N = 32, and for 90% "power" I get N = 35, while all along I find N = 27 to be sufficient.

Can you please help me understand when should I use the simple direct method coming from the formula and when should I use this approach by SAS (and probably others)?

Answer 1

(I don't have access to SAS, and don't get the same numbers as you in R or GPower.)

You and SAS are asking slightly different questions. You are asking what sample size you need when your mean is 2, to get p = 0.05.

SAS is telling you how likely it is that your result will be significant, given that the population mean is 2.

When you sample from a population where the mean is 2, you will sometimes get a higher value, and sometimes get a lower value, than 2. Both of these are equally likely. If the mean is exactly 2, then p will be 0.05, but the mean is less than 2, p will be greater than 0.05, and if the mean is greater than 2, then p will be smaller than 0.05. In other words, you have a 50% chance of getting a significant result. What you have worked out is the sample size to get 50% power - a 50% chance of a significant result.

Most people want more than 50% power. With a larger sample size, a value smaller than 2 will still be significant. SAS is saying that if you want a significant result with 80% chance, you're going to need a bigger sample.

Answer 2

This is not an answer. Did you try the following codes from the SAS manual? The SAS codes are for two sample t test. The first PASS manual you listed is for one sample test. The second PASS manual is for a tolerance interval which I think quite different from the ordinary confidence intervals.

proc power;
twosamplemeans test=diff
meandiff = 5 6
stddev = 12 18
alpha = 0.05 0.1
ntotal = 100 200
power = .;
run;