Can chi square be used to compare proportions?

Question

I've read that the chi square test is useful to see if a sample is significantly different from a set of expected values.

For example, here is a table of results of a survey regarding people's favourite colours (n=15+13+10+17=55 total respondents):

red,blue,green,yellow

15,13,10,17

A chi square test can tell me if this sample is significantly different from the null hypothesis of equal probability of people liking each colour.

Question: Can the test be run on the proportions of total respondents who like a certain colour? Like below:

red,blue,green,yellow

0.273,0.236,0.182,0.309

Where, of course, 0.273+0.236+0.182+0.309=1.

If the chi square test is not suitable in this case, what test would be? Thanks!

Edit: I tried @Roman Luštrik answer below, and got the following output, why am I not getting a p-value and why does R say "Chi-squared approximation may be incorrect"?

> chisq.test(c(0,0,0,8,6,2,0,0),p = c(0.406197174,0.088746395,0.025193306,0.42041479,0.03192905,0.018328576,0.009190708,0))

    Chi-squared test for given probabilities

data:  c(0, 0, 0, 8, 6, 2, 0, 0) 
X-squared = NaN, df = 7, p-value = NA

Warning message:
In chisq.test(c(0, 0, 0, 8, 6, 2, 0, 0), p = c(0.406197174, 0.088746395,  :
  Chi-squared approximation may be incorrect

Answer 1

Correct me if I'm wrong, but I think this can be done in R using this command

> chisq.test(c(15,13,10,17))

    Chi-squared test for given probabilities

data:  c(15, 13, 10, 17) 
X-squared = 1.9455, df = 3, p-value = 0.5838

This assumes proportions of 1/4 each. You can modify expected values via argument p. For example, you think people may prefer (for whatever reason) one color over the other(s).

> chisq.test(c(15,13,10,17), p = c(0.5, 0.3, 0.1, 0.1))

    Chi-squared test for given probabilities

data:  c(15, 13, 10, 17) 
X-squared = 34.1515, df = 3, p-value = 1.841e-07

Answer 2

Using the extra information you gave (being that quite some of the values are 0), it's pretty obvious why your solution returns nothing. For one, you have a probability that is 0, so :

• $e_i$ in the solution of Henry is 0 for at least one i
• $np_i$ in the solution of probabilityislogic is 0 for at least one i

Which makes the divisions impossible. Now saying that $p=0$ means that it is impossible to have that outcome. If so, you might as well just erase it from the data (see comment of @cardinal). If you mean highly improbable, a first 'solution' might be to increase that 0 chance with a very small number.

Given :

X <- c(0,0,0,8,6,2,0,0)
p <- c(0.406197174,0.088746395,0.025193306,0.42041479,0.03192905,0.018328576,0.009190708,0)

You could do :

> p2 <- p + 1e-6
> chisq.test(X,p2)

        Pearson's Chi-squared test

data:  X and p2 
X-squared = 24, df = 21, p-value = 0.2931

But this is not a correct result. In any case, one should avoid using the chi-square test in these borderline cases. A better approach is using a bootstrap approach, calculating an adapted test statistic and comparing the one from the sample with the distribution obtained by the bootstrap.

In R code this could be (step by step) :

# The function to calculate the adapted statistic.
# We add 0.5 to the expected value to avoid dividing by 0
Statistic <- function(o,e){
    e <- e+0.5
    sum(((o-e)^2)/e)
}

# Set up the bootstraps, based on the multinomial distribution
n <- 10000
bootstraps <- rmultinom(n,size=sum(X),p=p)

# calculate the expected values
expected <- p*sum(X)

# calculate the statistic for the sample and the bootstrap
ChisqSamp <- Statistic(X,expected)
ChisqDist <- apply(bootstraps,2,Statistic,expected)

# calculate the p-value
p.value <- sum(ChisqSamp < sort(ChisqDist))/n
p.value

This gives a p-value of 0, which is much more in line with the difference between observed and expected. Mind you, this method assumes your data is drawn from a multinomial distribution. If this assumption doesn't hold, the p-value doesn't hold either.

Answer 3

Yes, you can test the null hypothesis: "H0: prop(red)=prop(blue)=prop(green)=prop(yellow)=1/4" using a chi square test that compares the proportions of the survey (0.273, ...) to the expected proportions (1/4, 1/4, 1/4, 1/4)

Answer 4

The chi-square test is good as long as the expected counts are large, usually above 10 is fine. below this the $\frac{1}{E(x_{i})}$ part tends to dominate the test. An exact test statistic is given by:

$$\psi=\sum_{i}x_{i}\log\left(\frac{x_{i}}{np_{i}}\right)$$

Where $x_{i}$ is the observed count in category $i$. $i\in \{\text{red, blue, green, yellow}\}$ in your example. $n$ is your sample size, equal to $55$ in your example. $p_i$ is the hypothesis you wish to test - the most obvious is $p_i=p_j$ (all probabilities are equal). You can show that the chi-square statistic:

$$\chi^{2}=\sum_{i}\frac{(x_{i}-np_{i})^{2}}{np_{i}}\approx 2\psi$$

In terms of the observed frequencies $f_{i}=\frac{x_{i}}{n}$ we get:

$$\psi=n\sum_{i}f_{i}\log\left(\frac{f_{i}}{p_{i}}\right)$$ $$\chi^{2}=n\sum_{i}\frac{(f_{i}-p_{i})^{2}}{p_{i}}$$

(Note that $\psi$ is the effectively the KL divergence between the hypothesis and the observed values). You may be able to see intuitively why $\psi$ is better for small $p_{i}$, because it does have a $\frac{1}{p_{i}}$ but it also has a log function which is absent from the chi-square, this "reigns in" the extreme values caused by small expected counts. Now the "exactness" of this $\psi$ statistic is not as an exact chi-square distribution - it is exact in a probability sense. The exactness comes about in the following manner, from Jaynes 2003 probability theory: the logic of science.

If you have two hypothesis $H_{1}$ and $H_{2}$ (i.e. two sets of $p_i$ values) that you wish to test, each with test statistics $\psi_{1}$ and $\psi_{2}$ respectively, then $\exp\left(\psi_{1}-\psi_{2}\right)$ gives you the likelihood ratio for $H_{2}$ over $H_{1}$. $\exp\left(\frac{1}{2}\chi_{1}^{2}-\frac{1}{2}\chi_{2}^{2}\right)$ gives an approximation to this likelihood ratio.

Now if you choose $H_{2}$ to be the "sure thing" or "perfect fit" hypothesis, then we will have $\psi_{2}=\chi^{2}_{2}=0$, and thus the chi-square and psi statistic both tell you "how far" from the perfect fit any single hypothesis is, from one which fit the observed data exactly.

Final recommendation: Use $\chi_{2}^{2}$ statistic when the expected counts are large, mainly because most statistical packages will easily report this value. If some expected counts are small, say about $np_{i}<10$, then use $\psi$, because the chi-square is a bad approximation in this case, these small cells will dominate the chi-square statistic.

Answer 5

The test statistic for Pearson's chi-square test is

$$\sum_{i=1}^{n} \frac{(O_i - E_i)^2}{E_i}$$

If you write $o_i = \dfrac{O_i}{n}$ and $e_i = \dfrac{E_i}{n}$ to have proportions, where $n=\sum_{i=1}^{n} O_i$ is the sample size and $\sum_{i=1}^{n} e_i =1$, then the test statistic is is equal to

$$n \sum_{i=1}^{n} \frac{(o_i - e_i)^2}{e_i}$$

so a test of the significance of the observed proportions depends on the sample size, much as one would expect.