Suppose I have measured the outcome variable A using a (psychophysical) test that determines the ability of a subject to discriminate between two stimuli with a certain difference (the variable X). It was a two-alternative forced choice task, meaning that the subjects choose in each trial if the stimuli were diferent, yes or no. Now, dependent on the variable X, the subjects scored at minimum 50% (meaning they were guessing at small x) up to 100% (always choosing correct alternative at large X). The outcome variable A, now, is the value of X where correct rates are 75% (midway between chance and 100% correct rates).

I have measured A on three points in time (before, during, after: t0, t1, t2). At t0 no practice was received on the task, at t1 ten minutes and at t2 20 minutes of practice was received (8 participants). A repeated measures ANOVA showed a significant effect of practice as the scores go up by around 33%.

Now, during testing I have sampled at random a condition in which the subjects could theoretically never have been able to do the task above chance level (very small X), which are referred to as 'catch trials'. It turns out that these guess rates (theoretically 50%, i.e., chance level) increased significantly, being chance at t0 (as expected), but steadily, and significantly increasing above 50% at t1 (60%) and t2 (70%). The most parsimonious explanation here is that subjects were doing the task using false cues. In other words, they noticed cues between the stimuli other than X. This means that the learning effect of 33% is an overestimate.

Now, I plotted the individual improvement percentage (average 33% as noted above) against the corresponding guess rate pooling all data across t0, t1, t2. This, indeed, showed that A correlated significantly with guess rate. I did an F-test on the slope, P~0.01 and a correlation coefficient (r2) = 0.25.

Can I, based on this data set, quantitatively correct the learning effect of 33% by the finding that it, at least in part, was caused by the subjects using false cues? In other words, can I reduce the 33% learning improvement by a certain factor to correct for the subjects using other cues?

Is it as simple as subtracting the false-cue improvement (20%) from the total improvement (33%). Or should I take the 0.25 correlation coefficient into account as well?

ps: as it is my first step in CrossValidated, please feel free to shoot at the question to improve it

  • $\begingroup$ I know this is an old question :-) This is a curious situation, I wonder what your experiment actually was and what cue subjects manged to pick up that was not X. In any case, I would probably go back to estimating A, instead of "correcting it" afterwards. I.e. you probably are estimating A by fitting a logistic function to each subjects performance (?), and you could modify this procedure to account for the increased baseline performance (at X=0). $\endgroup$ – amoeba says Reinstate Monica Jun 27 '17 at 11:46
  • $\begingroup$ Apart from that, it's not very nice that you first find A for each subject, and then do further tests. It would be much neater to put everything into one statistical model, and fit mixed-model logistic regression directly accounting for practiceTime. But this becomes more involved, and I am guessing you have already finished this project anyway. $\endgroup$ – amoeba says Reinstate Monica Jun 27 '17 at 11:48
  • $\begingroup$ Hi Alice! I am wondering if you noticed my comments above? I can imagine that you are not interested in having this discussion here anymore, but then perhaps delete this question? $\endgroup$ – amoeba says Reinstate Monica Jul 3 '17 at 7:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.