5
$\begingroup$

I would like to calculate the power of a Chi-Square test for goodness-of-fit as a function of sample size for a specified alpha-value (say 0.01). Specifically, I am referring to power as the probability that the test will correctly reject the null hypothesis. In most places I look, I can find only vague references that this can be done and/or am referred to software. It seems to me that this should not be difficult. The specific question I would like to be able to answer (without using some sort of opaque software) follows:

Given an $\alpha$ value, for what sample size, $n$, can I expect a particular power, $\beta$?

Even better, how might I be able to calculate an ROC?

Improve this question
$\endgroup$
1
  • $\begingroup$ See this Q & A for P-value and power. $\endgroup$
    – BruceET
    Commented Jun 15, 2019 at 1:03

1 Answer 1

Reset to default
5
$\begingroup$

Does R fall into your idea of opaque software? If you are interested in this sort of calculation I would strongly recommend you use a stats package of some sort - and probably R in particular. In R, package pwr provides the function pwr.chisq.test which answers your question for you.

It isn't quite as simple as saying "for each sample size, what is the power of my test?" because as well as sample size, there is the question of the size of the effect in the underlying population you are inferring to. eg if there is a massive effect, then even a very small sample has a high power. As the effect gets smaller, you need a bigger sample size for the same power.

The documentation for pwr.chisq.test refers to Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale,NJ: Lawrence Erlbaum.

Also, a quick google search comes up with this reference (lecture 25), which shows that under the alternative hypothesis the test statistic has (asymptotically) a non-central Chi square distribution and provides a way to estimate the non-centrality parameter for a given alternative hypothesis.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Ideally, what I would like is a good reference. I do not want to use software without a very solid fundamental understanding of what I'm doing. $\endgroup$
    – cyrus1.618
    Commented Mar 29, 2012 at 16:51
  • $\begingroup$ I understand that effect size plays into it as well. My issue is that I can't find a satisfactory explanation for this calculation that does not at some point simply fall back and refer to software. It seems to me that this is something that should not be too difficult. I suppose this begs a further question: why is it so difficult to find a good explanation? Is it because it is considered too elementary? Thanks! $\endgroup$
    – cyrus1.618
    Commented Mar 29, 2012 at 16:54
  • $\begingroup$ ok, I've added some references $\endgroup$
    – Peter Ellis
    Commented Mar 29, 2012 at 18:41
  • $\begingroup$ Thanks for the references. The url reference is tangentially related (my distributions are not multinomial) - but it does illustrate an approach. Upon first glance in Cohen - it appears as if as soon as the text gets to power analysis of the chi-square test, it immediately jumps to tables without explanation of the derivation of the tables. I'll have to give a more thorough read through. Thanks for the references! $\endgroup$
    – cyrus1.618
    Commented Mar 29, 2012 at 23:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.