Suppose I have measured the outcome variable A using a (psychophysical) test that determines the ability of a subject to discriminate between two stimuli with a certain difference (the variable X). It was a two-alternative forced choice task, meaning that the subjects choose in each trial if the stimuli were diferent, yes or no. Now, dependent on the variable X, the subjects scored at minimum 50% (meaning they were guessing at small x) up to 100% (always choosing correct alternative at large X). The outcome variable A, now, is the value of X where correct rates are 75% (midway between chance and 100% correct rates).
I have measured A on three points in time (before, during, after: t0, t1, t2). At t0 no practice was received on the task, at t1 ten minutes and at t2 20 minutes of practice was received (8 participants). A repeated measures ANOVA showed a significant effect of practice as the scores go up by around 33%.
Now, during testing I have sampled at random a condition in which the subjects could theoretically never have been able to do the task above chance level (very small X), which are referred to as 'catch trials'. It turns out that these guess rates (theoretically 50%, i.e., chance level) increased significantly, being chance at t0 (as expected), but steadily, and significantly increasing above 50% at t1 (60%) and t2 (70%). The most parsimonious explanation here is that subjects were doing the task using false cues. In other words, they noticed cues between the stimuli other than X. This means that the learning effect of 33% is an overestimate.
Now, I plotted the individual improvement percentage (average 33% as noted above) against the corresponding guess rate pooling all data across t0, t1, t2. This, indeed, showed that A correlated significantly with guess rate. I did an F-test on the slope, P~0.01 and a correlation coefficient (r2) = 0.25.
Can I, based on this data set, quantitatively correct the learning effect of 33% by the finding that it, at least in part, was caused by the subjects using false cues? In other words, can I reduce the 33% learning improvement by a certain factor to correct for the subjects using other cues?
Is it as simple as subtracting the false-cue improvement (20%) from the total improvement (33%). Or should I take the 0.25 correlation coefficient into account as well?
ps: as it is my first step in CrossValidated, please feel free to shoot at the question to improve it