Still trying to grasp the right way to calculate with statistical significance. I believe to calculate with statistical significance, you must know your total population size first - but why does all of the A/B Test result calculator does not need a Population Size input? Am I missing something here.
2
-
1$\begingroup$ It's unclear what you mean by "all of the A/B ... calculator." Evidently, whatever calculators you are using assume there is no population (there's just a process that can yield an arbitrarily large number of observations) or else that the population is huge compared to any sample sample size you might contemplate. $\endgroup$– whuber ♦Commented Feb 2, 2020 at 16:44
-
1$\begingroup$ Population size or sample size? $\endgroup$– DaveCommented Feb 2, 2020 at 18:41
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
The population size is almost always irrelevant, unless the sample size is close to the population size. There is a finite population correction factor:
$$ \sqrt{\frac{N-n}{N-1}} $$
where N is the population size and n is the sample size. Plugging in various values of N and n shows that, usually, this factor is tiny. Therefore, most power analysis programs will ignore it.
answered Feb 11, 2020 at 13:05