2
$\begingroup$

I need to compare proportions of difference scores between 2 (unequal n) samples (males vs. females) on 2 different measures. I want to enter the difference scores into contingency table for chi-square type analysis.

I subtract male-female scores for each of the 2 measures

R1C1 = Male"Yes"MeasureA - Female"Yes"MeasureA 
R1C2 = Male"Yes"MeasureB - Female"Yes"MeasureB 
R2C1 = Male"No"MeasureA - Female"No"MeasureA  
R2C2 = Male"No"MeasureB - Female"No"MeasureB

When I use this method for getting difference scores, not surprisingly, some values are negative, which prevents me from doing chi-square. Is there a way to transform the data to do away with the negative values but preserve the proportions? For example, I was wondering if it would be acceptable to just square all difference scores, and then do chi-square?

So here's a bit more detail. I am investigating a measure of sexual experiences. The original measure asks respondents to indicate whether or not they've experienced a variety of sexual encounters. The survey has 2 parallel versions -- one for females (asking about sexual victimization) and one for males (asking about perpetration). research has shown that, when given the original measure, females indicate ~2/3s increased rates of victimization, than males reported rate of perpetration. I have created a modified version of the survey (for both male and female versions) and I have hypothesized that this modified version will decrease the discrepancy rate between female/victims and male/perpetrators rates of responding.

I have an unequal number of males and females. each participant was given both versions of the survey (original and modified), the original was given first. I have collapsed the response data to be dichotimous -- so either "yes" ([female]i have been raped/[male]i have raped someone) or "no" ([female]i have never been raped/[male]i have never raped anyone).

So, what i need is a way to determine if the male-female discrepancy ratio on the original measure is significantly different from the male-female discrepancy ratio of the modifed version.

further additional info. I have already run paired sample t-tests and determined that male report rates on the modified versin are significantly higher than on the original -- female report rates are not significantly different across versions. So i know that the discrepancy is reduced (because male reports increased and females did not) but I'm looking for a direct way to compare the difference scores/proportions between measures.

$\endgroup$
  • 3
    $\begingroup$ This isn't at all clear but a fair guess is that you are very confused. Chi-square tests at introductory level work on counts or equivalent (observed frequencies and expected frequencies). Best to back up and tell us more about the substantive problem and show us some raw data. $\endgroup$ – Nick Cox Jul 19 '13 at 17:10
  • $\begingroup$ @NickCox thanks for your response, i appologize for lacking clarity. i have added some more infromation to the question that will hopefully make my predicament more clear. Any advice? thanks! $\endgroup$ – Walter Jul 19 '13 at 19:24
  • 4
    $\begingroup$ Walter, that's a great improvement (+1). Instead of proposing a solution that puzzles us, you have explained your problem clearly in your own terms. That helps us understand your needs better and broadens the scope and potential value of the answers you might receive. $\endgroup$ – whuber Jul 19 '13 at 19:29
2
$\begingroup$

You say you did paired t-tests on the original data, before dichotomizing it, and that males increased significantly from the old form to the new but the female change was not significant. Unfortunately, that can not be taken as showing that the male change was bigger than the female change. You need to do an independent-groups t-test on the two sets of change scores. (Better yet, you could replace all the t-tests by confidence intervals for the corresponding means and mean differences, which would give you more information.)

For the dichotomized data, the situation is similar.
You have two contingency table, one for males and one for females.

Males      
        Yes   No     Total
  Yes   Myy   Myn    My.
   No   Mny   Mnn    Mn.

Total   M.y   M.n    M.. = M = total number of Males

Females      
        Yes   No     Total
  Yes   Fyy   Fyn    Fy.
   No   Fny   Fnn    Fn.

Total   F.y   F.n    F.. = F = total number of Females

For each table, the analog of the paired t-test is the McNemar test,
http://en.wikipedia.org/wiki/McNemar%27s_test

I know of no simple standard test of the difference between the changes in endorsement rates, but if all of Myn, Mny, Myy+Mnn, Fyn, Fny, Fyy+Fnn are "large" then an asymptotic test might be justified.

$\endgroup$
  • $\begingroup$ Thank you so much. Tell me, what do you think of the idea of combining male and female scores (e.g. Myy + Fyy; Myn+Fyn; Mny+Fny; Mnn+Fnn) and then doing a McNemar test? Do you think by combining the scores in this way i could address the issue of male/female proportional difference across surveys? Thanks!!!! Walter $\endgroup$ – Walter Jul 22 '13 at 20:21
1
$\begingroup$

You wrote

I have hypothesized that this modified version will decrease the discrepancy rate between female/victims and male/perpetrators rates of responding.

To test this I would recommend a slightly different approach to the proportions you said you want to use. If I understood you correctly your hypothesis would be confirmed if either male respondents answer "Yes" more often in the second than in the first questionnaire, or female respondents answer "No" more often in the second than in the first questionnaire. So these are two sub-hypotheses and either of these or both could be true.

So I would recommend doing two chi-square tests:

  1. Male questionnaire 1 yes-no vs. male questionnaire 2 yes-no. For example in Q1 10 of 50 answered Yes, and in Q2 20 of 50 answered Yes: (10;40) vs. (20;30).
  2. Female questionnaire 1 yes-no vs. male questionnaire 2 yes-no. For example in Q1 39 of 50 answered Yes, and in Q2 31 of 50 answered Yes: (39;11) vs. (31;19).

This would have the advantage that you can test if both females and males, only one or none of them are affected by your modifications to the questionnaire.

Do you think this would answer your questions?

$\endgroup$
  • $\begingroup$ thanks for the helpful suggestion. I edited my original post and added a bit more info to help further clarify. Could you maybe elaborate just a bit more about how your 2 chi-square proceedure could answer my question of whether or not the male-female discrepancies are significantly different across survey version? thank you sooo much! $\endgroup$ – Walter Jul 19 '13 at 20:23
  • $\begingroup$ You're welcome! The chi-square tests I suggested would do something similar to the t-tests you mention in your edit. I think there may be an advantage in using chi-square because this works on the actual counts you have and not on proportions like the t-tests. But it's not the direct way you were looking for. Maybe this direct way isn't necessary? For me as a reader it would be more informative to read that your modification affected males but not females in the way you intended - instead of that it changed the proportion. $\endgroup$ – robert Jul 19 '13 at 20:29
  • $\begingroup$ I noticed you haven't accepted any of the answers to the (other) questions you answered so far. Once you have received a (partially) satisfactory answer, please choose the one you prefer by accepting it/clicking on the check mark to the left of the answer. $\endgroup$ – robert Jul 20 '13 at 11:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.