Can you reproduce this chi-squared test result? Over at Skeptics.StackExchange, an answer cites a study into electro-magnetic hypersensitivity:


*

*McCarty, Carrubba, Chesson, Frilot, Gonzalez-Toledo & Marino, Electromagnetic Hypersensitivity: Evidence for a Novel Neurological Syndrome International Journal of Neuroscience, 00, 1–7, 2011, DOI: 10.3109/00207454.2011.608139.


I am dubious about some of the statistics used, and would appreciate some expertise in double-checking that they are used appropriately.
Figure 5a shows the results of a subject attempting to detect when an electromagnetic field generator was turned on.
Here is a simplified version:
 Actual:   Yes  No
Detected:
  Yes       32  19
  No       261 274

They claim to have used a chi-squared test, and found significance (p < 0.05, without stating what p is.)

The frequencies of the somatic and behavioral responses in the presence and absence of the field were evaluated using the chi-square test (2 × 2 tables) or the Freeman–Halton extension of the Fisher exact probability test (2 × 3 tables; Freeman & Halton,
  1951).

I see several problems. 


*

*They excluded some of the data - see Table 5b - where they left the device off for long periods. I cannot see the justification in separating that data.

*They seem to be claiming the result is statistically significant when the actual device was on, but not when it wasn't. (I may be misreading this; it isn't clear.) That's not a result that the chi-squared test can give, is it? 

*When I have tried to reproduce this test with an on-line calculator I have found it to be statistically insignificant.
This is my real question: Am I right in saying this?: A two-tailed, chi-squared test using Fisher's Exact Test is the right way to analyze this data AND it is NOT statistically significant.
 A: It seems to me that there are three things wrong with the conclusion. 
First, as @caracal said: They are reporting "significance" using a one-tailed test, without saying that they are doing so. Most people, I think, recommend using two-tailed tests almost always. Certainly it is not ok to use a one-tail test without saying so.
Second, the effect is tiny. When there was a signal, the subject (there was only one) detected it 11% of the time (32/293). When there was no signal, she detected a signal 6.5% of the time. That difference seems pretty small. And the subject was not able to detect the signal 89% of the time!
Third, as @oddthinking pointed out, there were some selective data reporting that were not properly explained or justified (I didn't read the paper carefully, so am simply repeating what was in the original post).
A: A Fisher exact test on the given table gives, per this code
actual <- c(rep("Y", 32), rep("N", 19), rep("Y", 261), rep("N", 274))
det <- c(rep("Y", 51), rep("N", 535))
table(det,actual) 
fisher.test(det,actual)

a p = 0.08
