# Differences in Fisher exact test statistic

I am trying to perform Fisher test on the following matrix:

269 118
55  48


Several free websites including R fisher.test() are returning p=0.0033, while vassarstats.net returns the output from the image.

Am I misinterpreting the data? Why the difference? • Although testing similar hypotheses, a chi-squared test is not identical to Fisher's exact test. Jun 11, 2019 at 18:37
• It is the Fisher test page, but I just realized the picture got cut off and you can't see Fisher result. I will fix that now. Jun 12, 2019 at 10:00

It is possible to perform the Fisher exact test manually in R to see which result is correct. To do this, we need to calculate the probabilities of each possible outcome of a contingency table with the same row and column totals as your table. (For computational reasons discussed here I will use log-probabilities instead.) The two-sided version of Fisher's exact test calculates the p-value as the sum of all the probabilities no greater than the probability of the observed contingency table.

#Input the observed contingency table and set parameters
DATA <- matrix(c(269, 118, 55, 48), nrow = 2);
m    <- sum(DATA[1, 1:2]);
n    <- sum(DATA[2, 1:2]);
k    <- sum(DATA[1:2, 1]);
maxx <- sum(DATA[1:2, 2]);

#Calculate log-probabilities over all possible outcomes
LOGPROBS <- rep(0, maxx+1);
for (x in 0:maxx) {
LOGPROBS[x+1] <- dhyper(DATA[1,1] - DATA[2,2] + x, m, n, k, log = TRUE); }

#Calculate p-value for Fisher exact test
LOWER   <- LOGPROBS[which(LOGPROBS <= LOGPROBS[DATA[2,2]+1])];
P_VALUE <- exp(matrixStats::logSumExp(LOWER));

P_VALUE;
 0.003255474


This manual calculation of the test gives the same p-value as the fisher.test function in R. I have also checked that the log-probabilities given here yield a total probability of one over all possibilities (to within a very small tolerance). So, based on this investigation, it appears to me that the calculation in R is correct, which suggests that there is some issue with the calculation at the website resource. (I suggest reading the documentation carefully to see if they are using some approximation method in their calculation.) Note that one possible source of calculation error is from arithmetic underflow problems if you try to compute the Fisher exact p-value without converting to log-probability space.

• Thank you, I have written to the authors about it. Jun 17, 2019 at 14:46

Interestingly, they have a page for Fisher's Exact as well: http://vassarstats.net/tab2x2.html If I multiply the one-tailed probability by 2, I get ~ .003, which is not the two-tailed probability they list. Best guess on the differences is the way they are estimating/calculating Fisher's Exact (VS is using the formula listed on that link I think, hard to know, while fisher.test is using a hypergeometric: "For 2 by 2 cases, p-values are obtained directly using the (central or non-central) hypergeometric distribution."

• The two tailed probability for a Fisher exact test is not double the one-tailed probability (in general, though it can happen). Jun 11, 2019 at 23:59
• I used that page, but the image got cut off. These are the probabilities I was asking for :) Still, the difference is quite significant, right? Jun 12, 2019 at 10:01

0.003255474 is the answer for Fisher's exact test where the test statistic is the probability of the observed table (i.e. we add up the probabilities of all the tables whose probabilities are less than or equal to that of the observed table) and 0.002423904 is the answer for Fisher's exact test where the test statistic is Pearson's Chi-square statistic (i.e. we add up the probabilities of all the tables whose Chi-square values would be as high or higher than the observed).