# What exactly is Daryl Bem's statistical test in “Feeling the Future”?

Daryl Bem published a paper called "Feeling The Future" (chronicled in Wired) in which he gives statistical evidence for extrasensory perception (ESP) using traditional statistical techniques in social psychology. This is controversial: is there ESP, or is stats in social psychology broken? Several authors have rebutted his claims, for example saying he did the statistics incorrectly.

I'm trying to understand one statistical claim in the paper.

He ran an experiment with 100 subjects. Each subject had 36 trials (12 neutral, 12 positive, 12 negative), in random order. Each trial was to "pre-identify" which (randomly chosen) curtain hid a picture. In the results, he says:

Across all 100 sessions, participants correctly identified the future position of the erotic pictures significantly more frequently than the 50% hit rate expected by chance: 53.1%, t(99) = 2.51, p = .01, d = 0.25.

In a footnote it says:

I set 100 as the minimum number of participants/sessions for each of the experiments reported in this article because most effect sizes (d) reported in the psi literature range between 0.2 and 0.3. If d = 0.25 and N = 100, the power to detect an effect significant at .05 by a one-tailed, one-sample t test is .80.

In another it says:

Unless otherwise indicated, all significance levels reported in this article are based on one-tailed tests and d is used as the index of effect size.

Which statistical test was he using? What is d? How exactly did he calculate "hit rate" across all users across all trials to be 53.1%?

My understanding is that a t-test would compare two averages. In this case is it an average of averages (average hit rate for a user, then average the user averages), or something else?

• It looks like he computed the percent correct for each subject and then did a t test comparing the mean percent to 50%. – David Lane May 25 '17 at 16:31
• Extraordinary claims require extraordinary evidence. Based on the results quoted here, it's not worth anyone's time to read further. – whuber May 25 '17 at 22:03
• @whuber I think that was Bem's point: using standard, accepted statistical techniques in psychology produces a result we don't believe. So likely there is something wrong with the standard of evaluation in psychology. – dfrankow Feb 10 '19 at 20:52

t-tests generally compare two means, but there is such a thing called a "one-sample t-test," where someone takes one mean and compares it to just any old constant number. In this case, Bem coded a miss as 0 and a hit as 1. In this case, the mean is the same thing as the percent correct.