Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I am trying to perform Fisher test on the following matrix:
269 118
55 48
Several free websites including R fisher.test() are returning p=0.0033, while vassarstats.net returns the output from the image.
Am I misinterpreting the data? Why the difference?
It is possible to perform the Fisher exact test manually in R to see which result is correct. To do this, we need to calculate the probabilities of each possible outcome of a contingency table with the same row and column totals as your table. (For computational reasons discussed here I will use log-probabilities instead.) The two-sided version of Fisher's exact test calculates the p-value as the sum of all the probabilities no greater than the probability of the observed contingency table.
#Input the observed contingency table and set parameters
DATA <- matrix(c(269, 118, 55, 48), nrow = 2);
m <- sum(DATA[1, 1:2]);
n <- sum(DATA[2, 1:2]);
k <- sum(DATA[1:2, 1]);
maxx <- sum(DATA[1:2, 2]);
#Calculate log-probabilities over all possible outcomes
LOGPROBS <- rep(0, maxx+1);
for (x in 0:maxx) {
LOGPROBS[x+1] <- dhyper(DATA[1,1] - DATA[2,2] + x, m, n, k, log = TRUE); }
#Calculate p-value for Fisher exact test
LOWER <- LOGPROBS[which(LOGPROBS <= LOGPROBS[DATA[2,2]+1])];
P_VALUE <- exp(matrixStats::logSumExp(LOWER));
P_VALUE;
[1] 0.003255474
This manual calculation of the test gives the same p-value as the fisher.test function in R. I have also checked that the log-probabilities given here yield a total probability of one over all possibilities (to within a very small tolerance). So, based on this investigation, it appears to me that the calculation in R is correct, which suggests that there is some issue with the calculation at the website resource. (I suggest reading the documentation carefully to see if they are using some approximation method in their calculation.) Note that one possible source of calculation error is from arithmetic underflow problems if you try to compute the Fisher exact p-value without converting to log-probability space.
Interestingly, they have a page for Fisher's Exact as well: http://vassarstats.net/tab2x2.html
If I multiply the one-tailed probability by 2, I get ~ .003, which is not the two-tailed probability they list. Best guess on the differences is the way they are estimating/calculating Fisher's Exact (VS is using the formula listed on that link I think, hard to know, while fisher.test is using a hypergeometric: "For 2 by 2 cases, p-values are obtained directly using the (central or non-central) hypergeometric distribution."
0.003255474 is the answer for Fisher's exact test where the test statistic is the probability of the observed table (i.e. we add up the probabilities of all the tables whose probabilities are less than or equal to that of the observed table) and 0.002423904 is the answer for Fisher's exact test where the test statistic is Pearson's Chi-square statistic (i.e. we add up the probabilities of all the tables whose Chi-square values would be as high or higher than the observed).
