The best way I found to do this was simply to simulate the data with 10k iterations of a binomial distribution for a chance-level learner, and calculate the d'.
I then took the 95th percentile as a cut off rate, with all d' above the 95th percentile being taken as statistically unlikely to be from a non-learner.
Very simple to run in R, especially with the
dprime command from the
neuropsychology package. I provide the code below, this for a study with 8 'yes' trials and 8 'no trials':
hit <- rbinom(10000,8,0.5)
miss <- 8-hit
fa <- rbinom(10000,8,0.5)
cr <- 8-fa
dprime.10k <- dprime(hit, miss, fa, cr)
quantile(dprime.10k$dprime,probs = (0.95))
This gave me a value of
0.8706082, although obviously you'd get a different value for a different number of trials, or if you wanted to go for a more conservative cutoff rate (I took the 5% rate under the assumption that I would not get any "anti-learners" who identified noise over signal).