comparing results of t tests obtained on the basis of different sample sizes I have variables a1 and a2 with 57 observations each, and variables b1 and b2 with 29 observations each.
I compare a1 with a2 by means of a paired-samples t-test, this gives p<0.001. Next, I compare b1 with b2 by means of a paired-samples t-test, this gives p>0.05 (actually p>0.10).
I compared the p-values, and concluded that I do find a significant effect for the larger set of 57 observations, and do not find a significant effect for the smaller set of 29 observations.
I am happy with that result, but got the criticism that the p-value of the first t-test cannot be compared to the one of the second test, because of the different sample sizes (57 versus 29).
What solution would you recommend to solve this problem?
 A: I agree with @ConnorG that you should use a standardized effect size, which is independent of sample size. However, the equation he presented is for between-subjects tests. Calculating a Cohen's d is trickier, but possible, doing paired-samples t-tests. There are multiple ways of calculating it, but I believe the simplest is Cohen's dz. See here for an illustration of multiple formulas.
You can then see which sample had a bigger effect size dz.
A: If your goal is to say that the difference you found in group a is greater than that in group b then you are going to need a formal test of the interaction. The easiest way to do this would be to do a repeated measures analysis of variance with group (a versus b) as the between participants factor and before ad after as the within subjects factor.
these three brief articles in the British Medical Journal explain in more detail what the issues are: http://www.bmj.com/content/313/7055/486
http://www.bmj.com/content/313/7060/808
http://www.bmj.com/content/313/7061/862
