Chi-squared vs. t-test for death rates (percentage data)

Question

I have data regarding death rates (DR) for 5 different groups of people (A-E) as shown below.

Groups	A	B	C	D	E
DR	1%	3%	2%	4%	3%
samples	200	800	300	100	700

I'm trying to find out observed differences between groups statistically significant. I know underlying population sizes so I can translate DR into death counts. Can I use a chi-squared test or t-test in this case or is there a better test/approach? What's your recommendation?

I'm relatively new to hypothesis testing and don't know all of them. I've found another topic with a similar question Calculating chi-squared values for percentages? but it was not exactly what I'm after. Your help would be much appreciated.

Answer 1

I don't see how a t test would be appropriate.

Test of equal proportions. In principle, the R procedure prop.test will test the null hypothesis that the five population proportions are equal against the alternative that they are not all equal.

This requires data in the form of death counts x and group sizes n, as below.

n = c(200, 800, 300, 100, 700)
pct = c(1, 3, 2, 4, 3)/100
x = n*pct
x
[1]  2 24  6  4 21

Unless the contrary is stated, prop.test uses the null hypothesis that all five groups have equal death rates.

prop.test(x, n)

        5-sample test for equality of proportions 
        without continuity correction

data:  x out of n
X-squared = 3.8952, df = 4, p-value = 0.4204
alternative hypothesis: two.sided
sample estimates:
prop 1 prop 2 prop 3 prop 4 prop 5 
  0.01   0.03   0.02   0.04   0.03 

Warning message:
In prop.test(x, n) : 
  Chi-squared approximation may be incorrect

The P-value is larger than $0.05 = 5\%,$ indicating that $H_0$ cannot be rejected at the 5% level of significance. However, the warning message raises the possibility that the P-value may not be accurate.

Chi-squared test. So it seems prudent to look at an essentially equivalent chi-squared test, procedure chisq.test in R. This procedure requires data in the form of a $2 \times 5$ contingency table, TAB computed below:

TAB = rbind(x, n-x);   TAB
  [,1] [,2] [,3] [,4] [,5]
x    2   24    6    4   21
   198  776  294   96  679

chisq.test(TAB, cor=F)  # Yates' correction declined

        Pearson's Chi-squared test

data:  TAB
X-squared = 3.8952, df = 4, p-value = 0.4204

Warning message:
In chisq.test(TAB, cor = F) : 
  Chi-squared approximation may be incorrect

Again here, we get a warning message. However, now we can look more carefully at the reason for it. The message is triggered when expected counts in the computation of the chi-squared statistic are below 5. Here the 4th group has such a 'small' expected count $2.714.$ Some statisticians might be willing to overlook just one expected count around $3$ when all the others exceed $5.$

chisq.test(TAB, cor=F)$exp
        [,1]      [,2]       [,3]      [,4] [,5]
x   5.428571  21.71429   8.142857  2.714286   19
  194.571429 778.28571 291.857143 97.285714  681

Chi-squared test with simulation. However, the chi-squared test as implemented in R's chisq.test allows for simulation of a more accurate P-value:

chisq.test(TAB, sim=T)

        Pearson's Chi-squared test 
         with simulated p-value 
         (based on 2000 replicates)

data:  TAB
X-squared = 3.8952, df = NA, p-value = 0.4028

Now it is clear that we cannot reject $H_0.$ [In a report for a non-statistical audience, I might quote the P-value $0.4204 > 0.05 = 5\%$ from the traditional version of the chi-squared test, mention the warning message calling out the one small expected count, and explain that a more accurate simulated P-value $0.4028$ leads to the same conclusion.]

Extended Fisher Exact Test. Finally for testing, it seems worth mentioning that the version of Fisher's Exact Test implemented in R, is able to handle contingency tables larger then $2 \times 2.$ [For tables with more cells, computation of Fisher's test may overwhelm available computer memory.]

fisher.test(TAB)

        Fisher's Exact Test for Count Data

data:  TAB
p-value = 0.3911
alternative hypothesis: two.sided

Notes: (1) Power. Another issue to consider is whether your sample sizes are large enough to have reasonable power of detecting actual differences among the five death rates. Suppose the true probabilities of death are as in your data, and you have $n = 500$ in each group. Then we can simulate the probability that prop.test will reject $H_0$ is about 75%. [You have an average of 420 subjects per group, with a minimum of 100 and a maximum of 800. I did not try to simulate your exact scenario because of the high probability of low expected counts. It is unfortunate that your smallest sample sizes are for groups with potentially the greatest difference in death rates.]

pct
[1] 0.01 0.03 0.02 0.04 0.03
set.seed(2021)
pv = replicate(10^5, prop.test(rbinom(5, 500, pct), rep(500,5))$p.val)
mean(pv <= .05)
[1] 0.75653

(2) Traditional Fisher test. The traditional version of Fisher's Exact Test for $2\times 2$ tables, uses an exact hypergeometric distribution instead of an approximate chi-squared distribution. For larger tables, fisher.test in R obtains the P-value by simulation.