5
$\begingroup$

I submitted a manuscript and have received the following comment I am not sure how to adequately answer. The context is that I have a table comparing a set of "scores" between men and women to see if there are any gender biases. I performed a Mann–Whitney U test to determine if there is a statistical difference between the male and female datasets. The reviewer said: "I would like to see effect size estimates with confidence intervals so readers can more easily interpret the gender disparities".

How can I compute an effect size estimate given the type of data I am dealing with? The data does not follow a normal distribution. For reference, I am using Python for my data analysis.

Edit: I see that one way to get the effect size from the Mann-Whitney U test is to take the U value and divided by the product of the two sample sizes. I can easily do this, although I do not entirely know how to interpret the results if this is indeed accurate.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
2
$\begingroup$

I think you may be looking for the Hodges-Lehmann two sample estimator outlined very briefly in this Wikipedia article which is the median of all the pari-wise differences between elements of each sample or possibly Somers' $D$ outlined in this Wikipedia article. I imagine either of these would be available in your preferred statistical software. There is also some discussion in this Q&A Mann-Whitney U-test: confidence interval for effect size

Improve this answer
$\endgroup$
2
  • $\begingroup$ Many thanks! I've seen the discussion in the last link you posted. That should be easy enough, although I need to calculate $z$ first, and neither Python or MATLAB (which I'm using) provide that for me easily. I'll definitely check out the other links. Thank you. $\endgroup$
    – Argon
    Dec 7, 2016 at 15:22
  • $\begingroup$ For those that stumble upon this post in the future, I found this Excel worksheet to be rather helpful. $\endgroup$
    – Argon
    Dec 10, 2016 at 22:25
1
$\begingroup$

While reviewer comments about statistical issues are often misguided, it seem in this case that there is a real need for some sort of summary or display of effect size. A summary or display is necessary for you, the analyst, as well as future readers of your work in order to make sense of any "statistical difference between male and female datasets." Otherwise how can you be confident that a statistically significant effect is scientifically interesting or important. Remember the first rule of testing: statistical significance is not the same as scientific significance.

You do not include enough information in the question for me to be sure that this is a relevant suggestion, but if your data are Likert scale scores then consider a simple graphical display of the proportions of respondents in each score category. If there is a large influence of sex on the responses then a major difference in the graphs should be apparent.

Improve this answer
$\endgroup$
1
  • $\begingroup$ I definitely want to provide as much information to the reader as possible. I just don't necessarily know how to do what the reviewer asked, although I'm getting closer. $\endgroup$
    – Argon
    Dec 7, 2016 at 15:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.