I want to test for a significant difference in the result (t-value or effect size) of two paired t-tests I am testing the quality of a measurement construct, where I propose that Measure A is better (i.e. more responsive to situational changes) than Measure B is.
I conduct a study in which I measure the construct at T1 and again at T2 (when it should be higher according to predictions), and I do so with each of the measures. So basically I have two conditions (whether a respondent fills out Measure A or Measure B), and each respondent fills out the measure they are assigned to twice.
Normally, I would be inclined to do this in a mixed ANOVA, with 1 between factor (which measure they fill out) and 1 within (T1 and T2) and test the interaction. However, this does not seem plausible as the DV is not the same in both conditions. Although they ostensibly should measure the same construct, they are different measures with different scales (one has 7 points, the other has 5 points).
I can easily do a paired t-test for Measurement A comparing T1 and T2, and get the t-value and effect size (with a confidence interval); and then also do so for Measurement B.
But how can I compare whether the effect size or t-value from Measurement A is actually different (predicted to be better) than the effect size or t-value from Measurement B? If the CI's of the effect sizes of Measurement A and B would not overlap I would of course be quite confident, but the effect is likely not that strong and I also find it annoying that I cannot think of a way how to test this for significance.
I would really appreciate some advice here!
 A: As a start, I think you have a prediction that could be tested with a contingency table chi-square test.
Your question says

measure the construct at T1 and again at T2 (when it should be higher
according to predictions)

and

I propose that Measure A is better (i.e. more responsive to
situational changes) than Measure B is

For each measure you can count the number of respondents who scored higher at T2 than T1, and the number of respondents who did not score higher at T2 than T1.  Each respondent thus contributes to exactly one cell of a 2x2 contingency table (measure A vs B by T2 higher vs not higher than T1).
Generally, your predictions suggest that the margin total for T2-higher should be higher than T2-not-higher, but the key hypothesis of interest is that, if measure A is better than measure B then there should be an interaction between measure and higher-or-not.
In particular, the hypothesis predicts that the cell count for measure A T2-higher will be especially large, and the cell count for measure A T2-not-higher will be especially small.
A: Thanks! This is indeed one solution. I do think there should also be other solutions, as this would make my measure rougher than it in reality is. Some respondents might not improve, some would by 0.2 scale points, and some by day 1 scale point. That precision is lost when dichotomizing the response.
