test for difference between two differences (proportions)

Question

I have 4 independent samples. Two groups A and B measured on time1 and two groups A and B measured on time2. For every group I calculate a proportion. I would like to compare the difference between the proportions over time for group A with the difference between the proportions over time for group B. To give an example:

• group A/time1 = 7%, group A/time2 = 14%; difference over time for group A = 14-7 = +7%
• group B/time1 = 12%, group B/time2 = 9%; difference over time for group B = 9-12 = -3%

(the samplesize in the 4 groups varies from 85 to 149) What test should I use to test whether the +7% is significantly different from the -3%?

Answer 1

Instead of computing manually the mean proportion in each group and comparing two numbers against each other, it would make more sense to run a test using all the information available, i.e. each individual's proportion. The variance around the mean is important when you are trying to compare mean differences between groups or over time.

Given that you have a single observation of your outcome variable ("proportion") for each participant, a simple test you could run to examine the mean differences over time, and how they differ between groups, is a repeated measures ANOVA. Should the assumptions of this test hold in your dataset, you 'd be interested in interpreting the Time*Group interaction. If this term is statistically significant you could conclude that in fact the mean difference that you observed by calculating the mean difference manually is a significant finding and not just a random occurence.