I have a subject who was giving 2-alternative forced choice responses for a task consisting of 80 trials. For each trial, there was a 50% chance for each of the two options to be correct. This means chance-level responding would, overall, be linked to a 50% mean accuracy across all trials. But since no subject's average accuracy will typically be precisely 50%, I am wondering what statistical concept allows me decide on a cut-off value for how far from 50% the mean accuracy would have to be to decide the subject was not responding randomly.
I know the binomial distribution can be used to model such responses, but I am not sure at which outcome I should look at. For instance, this page computes the binomial and cumulative probabilities associated with an experimental session of a certain
- Probability of success on a single trial
- Number of trials
- Number of successes (x)
My subject responded correctly on 51 out of the 80 trials, i.e. 63.75% correct. This gives a Cumulative Probability of P(X < 51)=0.99. How should I interpret this Cumulative Probability in plain English, in order to answer the question of whether or not the subject was responding randomly (at chance)? Would it not be quite arbitrary to decide, based on such a distribution, whether or not he was in fact understanding and doing his best at the task?! It's not like a simple % mean accuracy corresponds, 1-to-1, to a level of understanding of / involvement in the task!