3
$\begingroup$

I'd like to find out if there is a statistical test that I could use to determine if a mean difference is significantly greater than another mean difference.

To determine if one mean is different from another mean, I would use a t-test, but that would not apply here because there are not actual distributions; instead, there are differences.

Below is a replicable example in R to help explain. 

x1 <- c(1,1,1,1,0,0,0,0,1,1,1,1,1)
x2 <- c(1,1,1,1,0,0,0,0,0,0,0,0,0)

x1_mean_diff <- mean(x1) - mean(x2)
x1_mean_diff #sig different with a t test

y1 <- c(1,1,1,1,0,0,0,0,1,1,1,1,1)
y2 <- c(1,1,1,1,0,0,0,0,0,1,1,0,0)

y1_mean_diff <- mean(y1) - mean(y2)
y1_mean_diff #NOT sig different with a t test

Basically, I'd like to see if x1_mean_diff is significantly greater than y1_mean_diff. Is there a way to do this?

Please let me know if I'm being unclear. Thanks!

Improve this question
$\endgroup$
5
  • $\begingroup$ A t-test is not appropriate here (see wiki about assumptions). Chi-square test comes to mind here, where you would compare if the observer number of 0/1 is expected or not. $\endgroup$
    – Roman Luštrik
    Commented Mar 30, 2012 at 23:02
  • 1
    $\begingroup$ I'm confused, because the difference in sample means is the sample mean of the differences. so why not do a $t$-test on whether the mean of x1 - x2 is different from that of y1 - y2? $\endgroup$
    – shabbychef
    Commented Mar 30, 2012 at 23:47
  • 1
    $\begingroup$ Are x1 and x2 paired (and y1 and y2 also)? You don't mention it but from the pattern of your sample data it seems likely. You would use a different test if they are paired than if they aren't. $\endgroup$
    – Aaron - mostly inactive
    Commented Mar 31, 2012 at 1:57
  • $\begingroup$ Actually I'm thinking that pairing would only matter if the x-differences were paired with the y-differences. $\endgroup$
    – rolando2
    Commented Mar 31, 2012 at 12:46
  • $\begingroup$ @ Shabbychef -- that is essentially what I would like to do. But I need a sample size to calculate a t-test, don't I? How can I do a t-test on a mean when it is a summary score and not a distribution? Typically when I calculate a t ratio, I need a distribution so that I can find a standard error and mean. I'm not sure how to do that here. $\endgroup$
    – Captain Murphy
    Commented Apr 3, 2012 at 0:26

2 Answers 2

Reset to default
4
$\begingroup$

I agree with @Roman and will expand on his point a little. Imagine a 2x2 table. The rows are x-context and y-context. The columns are "Difference=0" and "Difference=1." Just populate the 4 cells with the appropriate integers. A garden-variety chi-square test of these data, and a corresponding phi correlation, will tell you how disproportionate the 2 rows are from each other. You may need to apply Yates' Correction, though, because one or more of your cells may have an expected (not actual) value <= 5.

Improve this answer
$\endgroup$
1
$\begingroup$

Assuming that you had data that were appropriate to use parametric tests on, an ANOVA with an interaction term will tell you if the size of the difference in one condition (1,2) depends on the level of the other variable (x,y). If the observed interaction has a low probability of occurring under an assumption that the null hypothesis is true you can conclude that is the case.

Or, you can just directly test the effects with a t-test of x1-y1 compared to x2-y2.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.