Over at Skeptics.StackExchange, an answer cites a study into electro-magnetic hypersensitivity:
- McCarty, Carrubba, Chesson, Frilot, Gonzalez-Toledo & Marino, Electromagnetic Hypersensitivity: Evidence for a Novel Neurological Syndrome International Journal of Neuroscience, 00, 1–7, 2011, DOI: 10.3109/00207454.2011.608139.
I am dubious about some of the statistics used, and would appreciate some expertise in double-checking that they are used appropriately.
Figure 5a shows the results of a subject attempting to detect when an electromagnetic field generator was turned on.
Here is a simplified version:
Actual: Yes No
Detected:
Yes 32 19
No 261 274
They claim to have used a chi-squared test, and found significance (p < 0.05, without stating what p is.)
The frequencies of the somatic and behavioral responses in the presence and absence of the field were evaluated using the chi-square test (2 × 2 tables) or the Freeman–Halton extension of the Fisher exact probability test (2 × 3 tables; Freeman & Halton, 1951).
I see several problems.
They excluded some of the data - see Table 5b - where they left the device off for long periods. I cannot see the justification in separating that data.
They seem to be claiming the result is statistically significant when the actual device was on, but not when it wasn't. (I may be misreading this; it isn't clear.) That's not a result that the chi-squared test can give, is it?
When I have tried to reproduce this test with an on-line calculator I have found it to be statistically insignificant.
This is my real question: Am I right in saying this?: A two-tailed, chi-squared test using Fisher's Exact Test is the right way to analyze this data AND it is NOT statistically significant.