I've been doing hypothesis tests to compare two proportions in a product. Although the sample size is supposed to be more than enough based on the population and $\alpha$ (type I error), the power of the experiment ($\beta$, type II error) is very low. Why is that?
1
-
$\begingroup$ Usual culprits: sample size, difference in means, standard deviations of means, $\alpha$. $\endgroup$– user2974951Commented Feb 19, 2019 at 9:24
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
There isnt a lot in your question to go on. So, you might not get much of an answer to your specific question.
A few thoughts. 1. Power = 1-beta, so if your beta is very low, then your power is actually very high. 2. The sample size calculation you performed should indicate a sample that is big enough to reject the null with a Type 1 error equal or less than alpha and Type 2 equal or less than Beta. But your sample size calculation is just an estimate based on an educated guess. Depending on what tests you are running and how the sample size was calculated you likely used an effect size from the literature or from a pilot study. If you are getting very different results than expected, it could either be that there was a flaw in the original sample size calculation, or there is always the chance that you have drawn an unrepresentative sample.
