Significant difference between difference scores of 2 groups on two measures?

Question

I am investigating a discrepancy between male and female self reports of sexual experiences. The original survey consists of a female version (asking about victimization) and a male version (asking about perpetration). Typically, when given the original version, female's reported rates of victimization are about 2/3s higher than male rates of perpetration.

I have modified the original survey (both male and female versions) in order to determine whether the wording of the modified version will have an impact on the female/victim--male/perpetrator discrepancy. One of my hypotheses is that the modified version will produce a narrower discrepancy between female reports of victimization, and male reports of perpetration.

I need to figure out what test (or series of tests) I can use to determine if there is a significant difference between the discrepancy rate of original survey, and the discrepancy rate of my modified version.

Additional info:

• males and females are not matched, and I have different sample sizes of males and females
• each subject was administered both versions (original and modified) of the survey, according to gender
• subjects answered the original survey first, and then were given the modified survey
• my data will be nominal -- e.g. "Yes" I've had this experience, or "no" I haven't had this experience.

Answer 1

To me, your hypothesis sounds like an interaction effect of the two factors "version" (of questionnaire) and "sex" (of participant). As the data are dependent / repeated measures, this should be taken into account by means of a multilevel aka hierachical aka mixed model. Instead of a linear model, you should use something like a logit link (cf. logistic regression) for your binary DV. Suppose your data are in long format, i. e., two rows for every participant, the first for the old version, the second for the new version. The DV is experience of crime 0/1. Data for 3 participants could like this:

     id sex version DV
[1,]  1   1       0  1
[2,]  1   1       1  0
[3,]  2   0       0  1
[4,]  2   0       1  1
[5,]  3   0       0  1
[6,]  3   0       1  0

According to your hypothesis, you predict a cross-level interaction between the level 1-predictor "version" and the level 2-predictor "sex" (in ANOVA / repeated measures terms, this is an interaction between a within- and a between-factor). If you code your variables like this sex: female=1, male=0; version: old=1, new=0, then you probably predict a positive beta of the interaction and you could test this one-sided. Code in R using the package lme4 could look like this (could be done with any multilevel software):

m1 <- lmer(DV ~ version*sex + (1|id), data=mydata, family=binomial(link="logit"))
summary(m1)

HTH