5
$\begingroup$

I have a survey where teachers are responding "yes" or "no" to lists of activities that they did in their classroom. There are separate checklists related to separate parts of the curriculum (e.g., reading, group activities, individual activities).

I want to find out if the proportion of "yes" responses is equal across the different parts of the curriculum. So the null hypothesis would be that the proportion of completed activities is equal across reading, group activities, and individual activities.

Would this be compared using ANOVA?

Thanks.

$\endgroup$
7
  • 1
    $\begingroup$ You wouldn't use anova, but you can do something similar with your count data. I am not sure if I have correctly understood you but you seem to be asking about homogeneity of proportions. One way to look at that is with a chi-square test. $\endgroup$
    – Glen_b
    Commented Sep 22, 2014 at 15:15
  • $\begingroup$ @Glen_b The reason for using proportions is that the number of activities is not the same for each checklist. So I could use Chi-Square to test the equivalence of proportions across the checklists? $\endgroup$
    – windy
    Commented Sep 22, 2014 at 15:17
  • $\begingroup$ The chi-square test on the counts actually does check for homogeneity of proportions. $\endgroup$
    – Glen_b
    Commented Sep 22, 2014 at 15:18
  • $\begingroup$ Right, thanks. But that seems like it would be biased due to the different number of items within each checklist, right? $\endgroup$
    – windy
    Commented Sep 22, 2014 at 15:19
  • $\begingroup$ Biased in what sense? I don't understand the problem you anticipate here. What do you think it's doing that would result in bias? $\endgroup$
    – Glen_b
    Commented Sep 22, 2014 at 15:20

1 Answer 1

5
$\begingroup$

You wouldn't usually use anova, but you can do something similar with your count data.

You seem to be asking about testing homogeneity of proportions. One way to look at that is with a chi-square test. Even though it operates on counts it is a test for equal proportions because it compares each count with the expected count if the proportions were equal. The further away the counts are from what you'd expect under equal proportions, the larger the chi-squared statistic is.

It sounds like low expected values may be an issue. It's possible to simulate from the null distribution rather than rely on the chi-square approximation in that case.

I could then follow up with z-tests to compare pairs of proportions, right?

You could certainly use a z-test, but I'd be more inclined to do 2x2 chi-square unless you care about a one-sided alternative. If you do use a z-test, you should be consistent about continuity correction (do it for both if you do it for either). Either z-tests or 2x2 chi-squares could be used for post-hoc tests with one of the basic multiple testing adjustments.

However, if you are after something more sophisticated, you might try Cox and Key (1993)$^{[1]}$.

If you have specific comparisons in mind you'd be better to specify contrasts.

If you have other independent variables to worry about, you could fit GLMs (specifically, one for a binomial response)

[1]: Cox, M.K. and Key, C.H. (1993),
"Post Hoc Pair-Wise Comparisons for the Chi-Square Test of Homogeneity of Proportions,"
Educational and Psychological Measurement, WINTER, 53:4, 951-962
http://epm.sagepub.com/content/53/4/951.abstract

$\endgroup$
2
  • $\begingroup$ @Glen_b Hi - thank you for the elaborative response. I'm looking to test proportions for A/B testing (add to cart, click through rate) between two groups. I'm unsure if chi-square is appropriate as I don't have integer counts, thus, I'm inclined to use z-proportion. Any thoughts on this? The proportions are calculates using integer values nonetheless (successful, v/s unsuccessful trials). $\endgroup$ Commented Jun 28, 2022 at 5:37
  • 1
    $\begingroup$ You know the denominator on that click through rate, right? So you have (or can calculate) both numerator and denominator, in which case you would have counts. $\endgroup$
    – Glen_b
    Commented Jun 28, 2022 at 23:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.