2
$\begingroup$

I have two categorical variables and was looking into doing a chi-square test. I then noticed I had some low frequencies in my contingency table and thought Fisher's Exact Test may be useful. I've now come full circle after doing some reading and want to use Pearson's Chi Squared with n-1 correction. Is there a way in R to run chisq.test with the n-1 correction (discussed here: Given the power of computers these days, is there ever a reason to do a chi-squared test rather than Fisher's exact test?)?

If not, how would I apply the correction to the output of Pearson's chi-squared?

Presuming a sample size of 80:

(80-1)/80 = 0.9875

Do I simply multiply the Chi-Squared statistic by 0.9875 and then use this value to derive the p value?

2.9687 * 0.9875 = 2.931591

1-pchisq(2.931591,4)

p = 0.569338

$\endgroup$
  • 1
    $\begingroup$ While it looks like your understanding is correct, given the chisq.test function already includes the ability to simulate to get the p-value (which works perfectly well down to very small expected values and can be made as accurate as you wish, simply by simulating more), why would you do this? $\endgroup$ – Glen_b Jun 3 '14 at 12:39
  • 1
    $\begingroup$ Purely because I've seen a few sources which recommend using it (having not heard of it before today). For instance, the link in my question and the link provided by Peter both suggest using the n-1 correction for where there are low cell frequencies. I'd be interested in the simulate method though. If you had any suitable links to further information on it (over and above the R help page) I'd be grateful. $\endgroup$ – Tumbledown Jun 3 '14 at 13:49
  • $\begingroup$ The Campbell 2007 paper about the n-1 adjustment, that is referenced in the answer you linked to, only deals with 2x2 contingency tables. In the example calculation you provided you have 4 degrees of freedom (df), which means you have a larger table (as I do). Can the n-1 adjustment be done irrespective of the table size? I would think so, because the chi-squared distribution takes care of the df, but this is just a gut feeling. Can anyone comment on this? Or would it be better to start a new question? $\endgroup$ – a tiger Apr 19 '17 at 11:20
2
$\begingroup$

According to this page the N-1 correction is very simple; just multiply $\chi^2$ by (N-1)/N. You could then use the pchisq function in R to get the right p value (the exact code would be, I believe, something like

newchisq = ((N-1)/N) * oldchisq
newp <- 1 - pchisq(newchisq, df)
| cite | improve this answer | |
$\endgroup$
  • 3
    $\begingroup$ Did you mean to multiply by (N-1)/N in the code, so that it matches your text? $\endgroup$ – Glen_b Jun 3 '14 at 11:00
  • $\begingroup$ Peter, the page you reference is wrong and that seems to have led you to propose two contradictory answers: one in the text and another in the code. I do not find it possible to determine which is correct because I can't figure out what you mean by the "old" chi-squared value. Could you fix this? $\endgroup$ – whuber Oct 3 '19 at 19:20
  • $\begingroup$ What is wrong with the page referenced by Peter? In my opinion it's correct. The formulas given there agree with e.g. Campbell (2007) [iancampbell.co.uk/twobytwo/background.htm , Section "Versions of the chi squared test"] and now all formulas in the Peter's answer are correct. $\endgroup$ – Wassermann Oct 3 '19 at 21:09
  • $\begingroup$ @Wasserman That page is quite vague, but the problematic part states " 'N -1' chi-square = Pearson chi-square x (N -1) / N." I understand the "N-1 chi-square" to be a statistic computed by dividing something by $N-1$ and the "Pearson chi-square" is intended to be the same thing divided by $N.$ Ergo, the correct procedure is to multiply the latter by $N/(N-1)$ but instead the formula multiplies by the reciprocal. $\endgroup$ – whuber Oct 3 '19 at 21:21
  • $\begingroup$ @whuber it's exactly as you wrote, just in the formulas there is multiplication by $N$ or $N-1$ not the division, hence the other form of the fraction. $\endgroup$ – Wassermann Oct 3 '19 at 21:36
1
$\begingroup$

Sorry for showing up so late to this party. I just happened across this discussion and saw that some of my notes were being discussed. I gather that this is the section of the notes that was causing some confusion for some folks:

Where Campbell describes replacing N with N -1, he is referring to this formula for Pearson's chi-square:

        chi-square = N(ad-bc)^2 / (mnrs)

where:

N is the total number of observations
a, b, c, and d are the observed counts in the 4 cells
^2 means "squared"
m, n, r, s are the 4 marginal totals

If one has the regular Pearson chi-square (e.g., in the output from statistical software), it can be converted to the 'N - 1' chi-square as follows:

           'N -1' chi-square = Pearson chi-square x (N -1) / N

If you want N(ad-bc)^2 / (mnrs) to become (N-1)(ad-bc)^2 / (mnrs), surely you must divide the whole expression by N and then multiply by (N-1).

I hope this clarifies things.

| cite | improve this answer | |
New contributor
Bruce Weaver is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.
$\endgroup$
0
$\begingroup$

Busing, Weaver and Dubois (2015) (http://onlinelibrary.wiley.com/doi/10.1002/sim.6808/full) give good advice on how to implement the N-1 chi-squared test in different software packages.

One possibility is to use the uncorrected Mantel-Haenszel chi-squared statistic which is equivalent to using N-1 chi-squared when you are analyzing a single 2x2 table.

In R, the uncorrected Mantel-Haenszel chi-squared can be obtained as follows:

stratum <- rep(1, N)
mantelhaen.test(variable1, variable2, stratum, correct = FALSE)

For the code to work as intended, the elements in the stratum vector must all have the same value.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ What does "stratum has to be a constant" mean? $\endgroup$ – Michael R. Chernick May 4 '17 at 15:21
  • $\begingroup$ I'm not sure of the correct language here. The Mantel-Haenszel test uses a vector of length N for identifying to which strata the corresponding elements in var1 and var2 belong. For the code to work as intended, the elements in this vector must all have the same value. $\endgroup$ – montriond May 4 '17 at 15:47
  • $\begingroup$ Now edited for better clarity. $\endgroup$ – montriond May 5 '17 at 12:24
0
$\begingroup$

Following the above discussions I wrote an R function implementing recommended policy from cited here Campbell (2007) paper which is:

  1. Where all expected numbers are at least 1, analyse by the ‘N − 1’ chi-squared test (the K. Pearson chi-squared test but with N replaced by N − 1).
  2. Otherwise, analyse by the Fisher–Irwin test, with two-sided tests carried out by Irwin’s rule (taking tables from either tail as likely, or less, as that observed).
campbell2x2.test <- function(t) {
  min_exp_val <- min(colSums(t))*min(rowSums(t))/sum(t)

  if (min_exp_val < 1) {
    # in Campbell's naming: Fisher–Irwin test by Irwin’s rule
    result <- fisher.test(t, alternative = "two.sided")
    result$method <- paste("Optimal 2x2 test according to Campbell(2007) recommendation\n\n",
                       paste("Minimal expected cell count: ", round(min_exp_val, 3), "\n\n", sep = ""),
                       paste("Performing", result$method)
    )
    return(result)
  } else {
    #  'N − 1' Pearson's Chi-squared test
    n1chisq.test <- function(t) {
      chisqtst <- chisq.test(t, correct = FALSE)
      N <- sum(chisqtst$observed)
  chisqtst$statistic = ((N-1)/N) * chisqtst$statistic
  chisqtst$p.value <- 1 - pchisq(chisqtst$statistic, chisqtst$parameter)
      chisqtst$method <- paste("'N-1'", chisqtst$method)
      return(chisqtst)
    }
    result <- n1chisq.test(t)
    result$method <- paste("Optimal 2x2 test according to Campbell(2007) recommendation\n\n",
                       paste("Minimal expected cell count: ", round(min_exp_val, 3), "\n\n", sep = ""),
                       paste("Performing", result$method)
    )
    return(result)
  }
}

Example usage:

campbell2x2.test(matrix(c(1, 5, 3, 2), nrow = 2))
| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.