Daryl Bem published a paper called "Feeling The Future" (chronicled in Wired) in which he gives statistical evidence for extrasensory perception (ESP) using traditional statistical techniques in social psychology. This is controversial: is there ESP, or is stats in social psychology broken? Several authors have rebutted his claims, for example saying he did the statistics incorrectly.
I'm trying to understand one statistical claim in the paper.
He ran an experiment with 100 subjects. Each subject had 36 trials (12 neutral, 12 positive, 12 negative), in random order. Each trial was to "pre-identify" which (randomly chosen) curtain hid a picture. In the results, he says:
Across all 100 sessions, participants correctly identified the future position of the erotic pictures significantly more frequently than the 50% hit rate expected by chance: 53.1%, t(99) = 2.51, p = .01, d = 0.25.
In a footnote it says:
I set 100 as the minimum number of participants/sessions for each of the experiments reported in this article because most effect sizes (d) reported in the psi literature range between 0.2 and 0.3. If d = 0.25 and N = 100, the power to detect an effect significant at .05 by a one-tailed, one-sample t test is .80.
In another it says:
Unless otherwise indicated, all significance levels reported in this article are based on one-tailed tests and d is used as the index of effect size.
Which statistical test was he using? What is d? How exactly did he calculate "hit rate" across all users across all trials to be 53.1%?
My understanding is that a t-test would compare two averages. In this case is it an average of averages (average hit rate for a user, then average the user averages), or something else?