1
$\begingroup$

I am recently learning Udacity A/B testing course with this note: http://rpubs.com/superseer/ab_testing.

As I learn the part about "Size vs Power trade-off", the video says "if we increase sample size, alpha value remains the same but beta value decreases". I get why beta value decreases as the distribution gets narrower, but does anyone know why the alpha value stay unchanged?

Thank you!

Improve this question
$\endgroup$
0

1 Answer 1

Reset to default
2
$\begingroup$

Alpha is set by the person doing the analysis. If they set alpha to 0.05, it stays 0.05 no matter what the sample size. Or another way to think about it, if the statistical test is designed correctly, and the analyst chooses 0.05 as the alpha, the probability that the test will return a type I error --- and the assumptions of the test are met --- is 0.05.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.