0
$\begingroup$

I'm often sharing data around performance and typically cut the data by gender to show differences around male vs female. So for example, I'll say 19% of males (which represents 550 out of 2800 total males) received an exceeds rating, while only 12% of females (which represents 300 out of 2500 total females) received an exceeds rating. I'm always asked "Is that difference significant?"

Which test would be preferred for measuring if the difference between males and females is significant? The available performance ratings are below, meets or exceeds, and gender is only male or female.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

A proportion test! Here is the result for your data

> dt=matrix(c(550,2800-550,300,2500-300),nrow=2,byrow=T)
> prop.test(dt)

    2-sample test for equality of proportions with continuity correction

data:  dt
X-squared = 56.727, df = 1, p-value = 5.006e-14
alternative hypothesis: two.sided
95 percent confidence interval:
 0.05658676 0.09627038
sample estimates:
   prop 1    prop 2 
0.1964286 0.1200000
Improve this answer
$\endgroup$
4
  • $\begingroup$ Awesome I thought about this too. What about the normality assumptions? How strict is it for this test. I’ve also heard about permutation testing as well. Thoughts? $\endgroup$
    – Ted Mosby
    Aug 22, 2019 at 14:22
  • $\begingroup$ This should be the same as a chi-square test of association. chisq.test(dt)... The chi-square approach may be better suited to examine the three categories simultaneously (that is, with a table with 3 columns and 2 rows). But then you get into the question of how to conduct a post-hoc investigation to see which categories among the three are different.... $\endgroup$
    – Sal Mangiafico
    Aug 22, 2019 at 15:16
  • $\begingroup$ yeah i thought of chi.sq but then i realized it would only tell me if the distributions are different, and not where the difference lies, which is the crucial piece. $\endgroup$
    – Ted Mosby
    Aug 22, 2019 at 15:25
  • $\begingroup$ Well, there're several ways to assess "where the differences lie" after a chi-square test. For example, you can look at standardized residuals or confidence intervals... For your purposes, three separate 2 x 2 analyses may be just as good (and easier to present)... It may be desirable to present your analyses as 2 x 2 chi-square tests of association, simply because your audience may be familiar with that, but you may be asked to explain what a "proportion test" is. $\endgroup$
    – Sal Mangiafico
    Aug 22, 2019 at 16:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.