0
$\begingroup$

I have the following problem.

I have a group of people where male and female members differ in their score on a certain scale. After an intervention both gender scored higher in a post test 6 months later. The increase in the score for women is roughly 9 % whereas men improvend only roughly 6 %.

What is the appropriate statistical method to test whether the womens' score improved (significantly) more than the mens score did? The increase for men as well as women in itself is stastistically sifnificant

Improve this question
$\endgroup$
4
  • $\begingroup$ This isn't the best design, but you could do an appropriate statistical test (t test or binomial, depending on how the score was measured) and specify the null hypothesis to be the original difference before the intervention. An ancova would be best however. Do you still have the original scores from pre-intervention? $\endgroup$
    – Demetri Pananos
    Nov 18, 2022 at 16:08
  • $\begingroup$ @DemetriPananos how would you improve the design? I don't see anything really lacking. $\endgroup$
    – AdamO
    Nov 18, 2022 at 16:10
  • 1
    $\begingroup$ @AdamO As Frank Harrell mentions here, changes from baseline probably not the best way to analyze these sorts of data. You want to model the raw score as a function of the pre score using ANCOVA $\endgroup$
    – Demetri Pananos
    Nov 18, 2022 at 16:29
  • 1
    $\begingroup$ @DemetriPananos that seems to be less a matter of the design, and more the problem of inappropriately formulating the response in the linear model. Pre-calculating the change and calling that the response introduces heteroscedasticity problems and inefficient inference when $\beta_0$ isn't exactly 1. $\endgroup$
    – AdamO
    Nov 18, 2022 at 16:53

3 Answers 3

Reset to default
2
$\begingroup$

Building on Demetri's point, the ANCOVA with a model term for difference-in-differences is the way to go:

$$ E[\text{Score}_{t=1} | \textrm{Sex}] = \beta_0 \text{Score}_{t=0} + \beta_1 \text{Sex} + \gamma \text{Score}_{t=0} \times \text{Sex}$$

Expands the paired t-test to allow an unequal difference in follow-up improvement for women versus men. Assuming the usual code for Sex is 0=Women, 1=Men, then the "excess" change (possibly negative) for men is estimated by $\gamma$. That is, if women improve by 10 points whereas men only improve by 5, $\gamma$ would be -5, and the significance test for the coefficient would test the null hypothesis that there is no difference in change between genders.

Improve this answer
$\endgroup$
0
$\begingroup$

If your sample size is sufficiently large, you could bootstrap each group, calculate each percent change, then subtract one percent change from the other. If both the CL95 and CL05 have the same sign (negative or positive), then the difference is "significant".

Improve this answer
$\endgroup$
0
$\begingroup$

If the scores are normally distributed, then paired t-test else Wilcoxon signed rank test. To know more refer to page 2 in the paper

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.