3
$\begingroup$

I have a question regarding the use of SPSS (i.e. its capability) to execute a Fisher's Exact test for large, sparse RxC contingency tables. I would like to test whether or not a certain correlation exist between my rows (diagnostic groups) and columns (laboratory tests) (see below).

Currently, I have a contingency table of 7 rows and 9 columns, that includes the data of 164 patients. Rows consist of certain diagnostic groups (e.g. different neurological diseases grouped together in a 'neurological disease' diagnostic group) while columns are the number of patients that have undertaken a certain laboratory test. However, multiple diagnostic groups have zero patients that underwent a certain test, and only 11.11% of the results are larger than 5 patients. As such, I can not use the Chi square test. Many suggest combining different rows and columns to evade this problem. However, in this case, I would like to avoid this since I am researching whether or not certain tests are associated with certain diagnostic groups, and by combining rows/columns, my research question would be in vain.

Therefore, I am looking for other possibilities to determine the presence of a correlation. One of my options would be to use a Fisher's exact test for R x C tables, since this test does not use the assumptions of the Chi squared test. I already know how to execute a Fisher's Exact test in SPSS for R x C tables. However, is it feasible to execute a Fisher's exact test for a table as large as 7x9, or should I use other statistical tests (and if yes, which tests do you recommend)?

Thanks in advance.

$\endgroup$
  • 1
    $\begingroup$ Have you tried executing it to check? $\endgroup$ – user2974951 Sep 18 '18 at 12:21
  • 1
    $\begingroup$ I am currently executing it, but it has already been going on for 5 straight hours (currently I am at iteration +- 5000). Therefore, I started wondering if it was even feasible or that I would better look at other options instead of waiting hours/days for a result of the Fisher Exact test. $\endgroup$ – jgodel0 Sep 18 '18 at 12:36
  • 3
    $\begingroup$ I don't know about SPSS but in r there is an argument to calculate an approximate p value using Monte Carlo simulations specifically for tables larger than 2 x 2. $\endgroup$ – user2974951 Sep 18 '18 at 12:41
  • $\begingroup$ Well, I have seen this option in SPSS but since I am a novice when it comes to statistics I was not sure whether or not you can use it a) instead of the Fisher Exact test, b) in large tables and c) in tables with multiple 0's and low counts. Is the approximate p-value with a Monte Carlo simulation just as 'justified' as a Fisher Exact test in a situation like this (large RxC table, low counts, multiple 0's)? $\endgroup$ – jgodel0 Sep 18 '18 at 12:49
  • 1
    $\begingroup$ The argument is merely a modification of the Fisher test but it is still done in the fashion of the Fisher test. Here is an extract from the r documentation In the r x c case with r > 2 or c > 2, internal tables can get too large for the exact test in which case an error is signalled. Apart from increasing workspace sufficiently, which then may lead to very long running times, using simulate.p.value = TRUE may then often be sufficient and hence advisable. $\endgroup$ – user2974951 Sep 18 '18 at 12:56
4
$\begingroup$

Transcribing the comments into an answer.

There should be an argument to calculate an approximate p value using Monte Carlo simulations specifically for tables larger than 2 x 2. The argument is merely a modification of the Fisher test but it is still done in the fashion of the Fisher test.

Here is an extract from the r documentation In the r x c case with r > 2 or c > 2, internal tables can get too large for the exact test in which case an error is signalled. Apart from increasing workspace sufficiently, which then may lead to very long running times, using simulate.p.value = TRUE may then often be sufficient and hence advisable.

$\endgroup$
1
$\begingroup$

Just to add to @user2974951 answer and respond to a question raised by the OP in comments.

The best way of seeing how many simulated $p$-values to generate is to try it. The code I used is pasted at the foot of this e-mail but what I did was to generate 63 random numbers from the Poisson with parameter $\lambda = 164 /63$ since the OP had a sample size of 164. I then ran the fisher.exact function in R setting the number of simulated $p$-values to 2000 (the default), 10000 and 100000. I repeated each of these 1000 times to get a picture of the Monte Carlo error (how variable they are. I present below the summaries.

# 2000
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
 0.5827  0.6052  0.6127  0.6129  0.6207  0.6482 
# 10000
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
 0.5982  0.6094  0.6127  0.6128  0.6160  0.6303 
# 100000
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
 0.6086  0.6119  0.6131  0.6130  0.6141  0.6188 

As can be seen there is a fair amount of Monte Carlo error still for 2000 but this has reduced to a small amount for higher values. Probably the most stable way is to run the function a few times for 2000 and take the median $p$-value.


The code I used is below. This does not take very long, seconds to run for the smaller numbers and slightly more than a minute for 100000 using 8 clusters on an I7 running Windows 7.

require(parallel)

set.seed(2009)
dat <- matrix(rpois(63, 164/63), nrow=7) # simulate some data
cl  <- makeCluster(8)                    # set up clusters

clusterExport(cl, "dat")
b2000   <- parSapply(cl, 1:1000, function(x)
                     fisher.test(dat, simulate.p.value=TRUE, B=2000)$p.value)
b10000  <- parSapply(cl, 1:1000, function(x)
                 fisher.test(dat, simulate.p.value=TRUE, B=10000)$p.value)
b100000 <- parSapply(cl, 1:1000, function(x)
                     fisher.test(dat, simulate.p.value=TRUE, B=100000)$p.value)
stopCluster(cl)
$\endgroup$
  • 2
    $\begingroup$ You can avoid most of this work by noting that a simulated p-value is an observation of a Binomial variable. From that you readily obtain an estimate of its standard error, which lets you determine (to a reasonable approximation) how many values to simulate in order to limit the simulation error to any small value you desire. $\endgroup$ – whuber Sep 21 '18 at 13:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.