0
$\begingroup$

I have a dataset of about 100,000 individuals, each of which is assigned a score between 1 and 5. Let's assume half are male and half are female, so 50,000 each.

I find that the average score for males is 3.9 whereas the average score for women is 3.6. How do I determine if gender has a statistically significant role? For reference, the scores do not follow a normal distribution.

Would a chi-squared test be appropriate by comparing this to the expected case of the results being identical for males and females?

Improve this question
$\endgroup$
4
  • $\begingroup$ Something to be wary of: if you do find that men and women have a significantly different distribution of scores, don't then immediately conclude that it's because women are worse at whatever task they're being scored on. The distributions of men and women that you have may well be different for some confounding reason. $\endgroup$
    – Danica
    Aug 26, 2016 at 3:08
  • $\begingroup$ Is the question specifically about whether the population means differ, or is it something else (e.g. do the distributions differ? Is the probability that a random male scores more than a random female different from 50%?)...? Do you have anything more than the averages like the original data? (you can still work with only the information you have mentioned, but if you have the data it's easier to justify to other people without a lot of explanation) $\endgroup$
    – Glen_b
    Aug 26, 2016 at 4:07
  • $\begingroup$ @Dougal We should assume the experiment is non-biased, unless otherwise. $\endgroup$
    – ABCD
    Aug 26, 2016 at 4:09
  • $\begingroup$ @StudentT I only meant the standard correlation-is-not-causation caveat, since this seems like a setting where misinterpretation of those results is easy. $\endgroup$
    – Danica
    Aug 26, 2016 at 6:10

3 Answers 3

Reset to default
1
$\begingroup$

This looks like a Wilcox rank sum test (Mann–Whitney U test) to me.

  • It's a non-parametric test
  • The null hypothesis is that the two samples come from the same population
  • Assuming your observations in male and female are independent
  • Your data is ordinal, for example 3.9 is better than 3.6

Please note with 50,000 sample size, you test will most likely be significant. You may want to take a smaller sample for the test.

Improve this answer
$\endgroup$
1
  • $\begingroup$ If the scores are being averaged, they were already assumed to be at least interval when they were added together (since that treats 2+3 the same as 4+1 for example) $\endgroup$
    – Glen_b
    Aug 26, 2016 at 4:06
0
$\begingroup$

I would suggest using the Kolmogorov-Smirnov Test to determine if these two datasets differ significantly. It is non-parametric, so this suits your dataset.

An advantage of the K-S test is that you can plot the cumulative fraction of the two groups and view these graphically.

Improve this answer
$\endgroup$
0
$\begingroup$

With approx n=50,000 in each sample, there's no point in a statistical test since it will most likely be statistically significant regardless of the meaning of the difference. Instead, I'd plot the two distributions (histograms and boxplots) and carefully consider what is the practical meaning of a difference of 0.3 between the two genders. Most importantly, the question is: what will this result be used for? Is it really the averages that you need to compare?

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.