Very simple chi-square question

I have a super simple chi-square in R question, but I'm struggling to wrap my head around it. The question was to take a random 20 people from the dataset, count males and females and then use chi-square test to accept/reject the null hypothesis that half of the population in the entire large dataset is men. So this is my data table:

How can I run chisq.test on just 2 numbers? Or should there be an 'expected' column with values of 10 for each row and then run the function? I haven't been able to find a similar chi-square problem on the internet.

• you have to use a test for a single proportion Commented Nov 11, 2022 at 6:15
• If this is a question from a textbook, course, or test used for a class or self-study, could you add the self-study tag ? Commented Nov 12, 2022 at 1:42
• Just consider, if all your sample were men, or women, you'd have only one number! :) Commented Dec 1, 2022 at 19:54

At the time of writing, the two answers suggest a binomial test. This is a good approach to assess a binomial set of counts, like in your question.

But there is also a chi-square goodness-of-fit test that can be used in these cases.

R has this built in to the chisq.test() function.

And it can be used when there are more than two categories. Or when the theoretical proportions aren't equal across categories.

That is,

Gender = c("Female", "Male")
Count  = c(12, 8)
chisq.test(Count)

Gender = c("Female", "Male", "Other")
Count  = c(12, 8, 6)
chisq.test(Count)

Race = c("American Indian", "Asian", "Black", "Pacific Islander", "White")
Count = c(10, 8, 16, 1, 24)
Theoretical = c(0.10, 0.15, 0.16, 0.0, 0.59)
chisq.test(Count, Theoretical)

Like a chi-square test of association, the chi-square goodness-of-fit test has a suggested minimum for expected values:

Gender = c("Female", "Male")
Count  = c(12, 8)
chisq.test(Count)$expected And you can extract the standardized residuals: Gender = c("Female", "Male", "Other") Count = c(12, 8, 6) chisq.test(Count)$stdres

There are exact and Monte Carlo approaches. You might look at the multinomial.test() function in the EMT package.

There are also multinomial confidence intervals. You might look at the MultinomCI() function in the DescTools package.

Often the best way to express effect size is to compare the expected proportions to the observed proportions (e.g. rcompanion.org/handbook/images/image301.png ).

Because there's some suggestion in the answers about which test may be better, below is the results for this example from a few different tests.

Without attempting justification as to which is more correct, in this case the p-values from the exact tests and Monte Carlo simulations are similar.

The uncorrected chi-square test is probably too liberal in this case. Though a Yates correction could be applied here too. (Though I don't know of an easy implementation in R for the goodness-of-fit chi-square test with Yates correction).

A = c(12, 8)

N = sum(A)

theoretical =c(0.5, 0.5)

#####################

binom.test(A, N)

### Exact binomial test
###
### number of successes = 12, number of trials = 20, p-value = 0.5034

chisq.test(A)

### Chi-squared test for given probabilities
###
### X-squared = 0.8, df = 1, p-value = 0.3711

chisq.test(A, simulate.p.value=TRUE, B=10000)

### Chi-squared test for given probabilities with simulated p-value (based on 10000 replicates)
###
### X-squared = 0.8, df = NA, p-value = 0.506

library(DescTools)

GTest(A, correct="yates")

### Log likelihood ratio (G-test) goodness of fit test
###
### G = 0.4517, X-squared df = 1, p-value = 0.5015

library(EMT)

multinomial.test(A, theoretical)

### Exact Multinomial Test
###
### Events    pObs    p.value
###     21  0.1201     0.5034

library(EMT)

multinomial.test(A, theoretical, MonteCarlo = TRUE, ntrial=10000)

### Monte Carlo Multinomial Test
###
### Events    pObs    p.value
###     21  0.1201     0.5026

I couldn't resist seeing how the Yates correction on the chi-square goodness of fit test would work out. With this correction, it's in line with the exact and Monte Carlo tests.

A = c(12, 8)

DF = length(A)-1

Exp = theoretical * N

YatesChisq = sum((abs(A-Exp)-0.5)^2/Exp)

pValue = pchisq(YatesChisq, DF, lower.tail=FALSE)

(data.frame(YatesChisq=round(YatesChisq, 2), pValue=round(pValue,4)))

### YatesChisq pValue
###       0.45 0.5023

Thanks to @utobi for pointing out that the Yates correction can be applied for a chi-square goodness-of-fit test when there are two categories

x = 12
n = 20
prop.test(x, n, correct=TRUE)

### 1-sample proportions test with continuity correction
### X-squared = 0.45, df = 1, p-value = 0.5023

Comparing the p-values from the chi-square goodness-of-fit test and the binomial test, where the sum of counts for two categories is 20.

G1 = 1:10
G2 = 20-G1

pChiSq = rep(NA, 10)

pBinom = rep(NA, 10)

for(i in 1:10){

pChiSq[i] = chisq.test(c(G1[i],G2[i]))$p.value pBinom[i] = binom.test(G1[i],(G1[i]+G2[i]))$p.value

}

(data.frame(Count1=G1, Count2=G2, pChiSq=round(pChiSq,5), pBinom=round(pBinom,5)))

### Count1 Count2  pChiSq  pBinom
###      1     19 0.00006 0.00004
###      2     18 0.00035 0.00040
###      3     17 0.00175 0.00258
###      4     16 0.00729 0.01182
###      5     15 0.02535 0.04139
###      6     14 0.07364 0.11532
###      7     13 0.17971 0.26318
###      8     12 0.37109 0.50344
###      9     11 0.65472 0.82380
###     10     10 1.00000 1.00000

Your problem can be seen as having an iid sample $$X_1,\ldots,X_n$$ with $$X_i\sim \text{Bernoulli}(\theta)$$, with say $$X_i=1$$ if the sample is female and $$X_i=0$$ otherwise. In this notation, $$\theta$$ is the probability of observing a female.

The aim is to test $$H_0:\theta=1/2$$ vs $$H_1:\theta\neq 1$$.

There are many ways to test $$H_0$$, e.g. through a hypothesis testing procedure or a confidence interval. Both can be obtained by noting that if $$\hat\theta = \bar X$$ is the sample proportion of females, then, under $$H_0$$

$$n\bar X \sim \text{Bin}(n,\theta_0),$$

which can be used to perform a test statistic or it may be inverted to obtain a confidence interval. There are many ways to perform such inversion and an exact R implementation is

> x <- 8
> n <- 20
> binom.test(x, n, p=0.50)

Exact binomial test

data:  x and n
number of successes = 8, number of trials = 20, p-value = 0.5034
alternative hypothesis: true probability of success is not equal to 0.5
95 percent confidence interval:
0.1911901 0.6394574
sample estimates:
probability of success
0.4

You might want to have a look also at the exactci package for other related approaches.

Another path may be to invoke the Central Limit Theorem, i.e.

$$T_n = \frac{\sqrt{n}(\bar X - \theta)}{\sqrt{\theta(1-\theta)}}\overset{d}{\to} N(0,1).\quad\quad(*)$$

Under $$H_0$$, $$(*)$$ is a pivotal quantity and can be used to build the approximate $$\alpha$$-level test statistic:

Reject $$H_0$$ if the observed value of $$T_n$$ is in absolute value greater than $$z_{1-\alpha/2}$$.

Now for large $$n$$, $$T_n^2$$ will be approximately $$\chi_1^2$$, thus another equivalent $$\alpha$$-level test statistics would be

Reject $$H_0$$ if the observed value of $$T_n^2$$ is greater than $$\chi_{1-\alpha}^2$$.

In R this can be performed (using a continuity correction for better accuracy) by

> x <- 8
> n <- 20
> prop.test(x, n, p=0.5)

1-sample proportions test with continuity correction

data:  x out of n, null probability 0.5
X-squared = 0.45, df = 1, p-value = 0.5023
alternative hypothesis: true p is not equal to 0.5
95 percent confidence interval:
0.1997709 0.6358833
sample estimates:
p
0.4
• +1, but note that the question asks specifically about the chi-square test. Commented Nov 12, 2022 at 2:00
• @SalMangiafico I expanded my answer to cover the case of the chi-square test. Commented Nov 12, 2022 at 20:22
• Oh, good. I didn't realize that prop.test() could be used for a chi-square goodness-of-fit test with Yates correction. Commented Nov 13, 2022 at 0:42

As with any table, the chi-squared statistic is the sum of $$(O-E)^2/E$$ where $$O$$ is the observed count in each cell and $$E$$ is the expected count.

In your case, there are two cells with observations $$12$$ and $$8.$$ The null hypothesis asserts that the expected counts are each $$1/2$$ times the sample size, $$E = 1/2\times 20 = 10.$$ Thus

$$\chi^2 = \frac{(12 - 10)^2}{10} + \frac{(8 - 10)^2}{10} = \frac{8}{10}.$$

The p-value (for the two-sided alternative to the null; namely, that the population proportion of males differs from $$1/2$$) is approximated by the right tail area of a chi-squared distribution. The one to use has $$2-1 = 1$$ degrees of freedom, because (a) you have two cells and (b) the null hypothesis specifies one parameter, leaving one left over. The p-value, computed with the R function pchisq(8/10, 1, lower.tail = FALSE), is 37%.

Some have contended this is the "wrong" test. Far from it, this is an excellent test. Part of the proof is to consider the distribution of the possible p-values under the null hypothesis. (The other part is to examine its power, but that would take us far afield.) Ideally, the p-values will be uniformly distributed between $$0$$ and $$1.$$ Because there are only $$11$$ distinct outcomes ($$0$$ and $$20$$ lead to the same decision, $$1$$ and $$19$$ to the same, and so on) the ideal is impossible to attain for any (non-randomized) decision procedure whatsoever. But how close can we come? Look at the actual distribution function:

The reference uniform distribution appears as the dashed red line. The plot at left shows that the null distribution of p-values comes as close as reasonably possible to the uniform reference distribution, at least insofar as we can see. (In the very rare cases where one of the cells is zero, I have set the p-value to zero.)

The plot at the right is the same, shown on log-log axes to reveal details for small p-values. From left to right are the outcomes $$0,20; 1,19; 2,18;$$ and so on. Except at the extreme left, where one of the cell counts is only $$0$$ or $$1,$$ the p-values are approximately uniformly distributed. Moreover, this failure at the left is minor: in almost any case you would correctly reject the null even though the p-value is a little larger than it ought to be.

A standard rule of thumb, by the way, is that you can trust the chi-squared test when all expected counts are $$5$$ or larger. In this case, both expected counts are $$10:$$ the rule gives good advice.

• BTW, the Yates correction makes the distribution of p-values slightly less uniform, not more uniform, and does almost nothing to improve the situation for the tiniest p-values.
– whuber
Commented Dec 1, 2022 at 17:39
• The plot of the null distribution provide great insight. I'm wondering how you determined the cumulative probabilities of the y-axis? Naively, I'd use the binomial distribution or simulation. Commented Dec 1, 2022 at 22:11
• @COOLSerdash Yes, I used the Binomial distribution. There are only 11 possibilities, making the calculation very fast and far easier than simulation.
– whuber
Commented Dec 2, 2022 at 14:09
• Thanks for your continued help, I appreciate it. Commented Dec 2, 2022 at 14:25
• (+1) Excellent explanation! Commented Dec 28, 2022 at 14:31

You are using the wrong test. While you could run chi squared test (with expected values of 10 and 10) and 1 degree of freedom, there is a much better way. You haven't found examples like this because you should use a binomial test instead.

Given the null hypothesis (and quite reasonable assumptions on the large size of the population and the sampling method) the number of females would have a binomial distribution B(20,0.5).

So you use the binomial distribution to find a (say) 95% acceptance region. The exact region depends on if you want a two-tail or one-tail test. But 6≤n≤14 works for me.

And then reject your null hypothesis if the observed value of female lies outside that region.

• Since a chi-squared test of $6,14$ would give a $p$-value of about $0.074$ and one of $5,15$ would give a $p$-value of about $0.025$, saying "you are using the wrong test" seems a little strong Commented Nov 11, 2022 at 20:54
• It is the wrong test. The chi square test is a useful approximation in many situations, but in this situation the exact binomial test is just as easy to use. This is like using the car to drive to a shop around the corner. THe car is useful in many situations, but when the destination is just round the corner, it is quicker to walk. So giving deriving instructions is not useful. Commented Nov 11, 2022 at 21:41
• In fairness, the question states, "The question was to ... use chi-square test to accept/reject the null hypothesis ... How can I run chisq.test on just 2 numbers?". So claiming that chi-square is the wrong test may not directly answer the question. Commented Nov 12, 2022 at 1:40
• To flesh out the comment by @Henry , I added Addendum 3 to my answer, comparing the p-value from chisq.test() and binom.test(). For the two-sided test, with an alpha of 0.05, and a sum of counts of 20, the conclusions for each of these tests would be the same. Commented Nov 12, 2022 at 1:58
• I know, but this is literally my school assignment and it requires us to use chiqs.test Commented Nov 13, 2022 at 18:52