I have variables a1 and a2 with 57 observations each, and variables b1 and b2 with 29 observations each.
I compare a1 with a2 by means of a paired-samples t-test, this gives p<0.001. Next, I compare b1 with b2 by means of a paired-samples t-test, this gives p>0.05 (actually p>0.10).
I compared the p-values, and concluded that I do find a significant effect for the larger set of 57 observations, and do not find a significant effect for the smaller set of 29 observations.
I am happy with that result, but got the criticism that the p-value of the first t-test cannot be compared to the one of the second test, because of the different sample sizes (57 versus 29).
What solution would you recommend to solve this problem?