# Manually calculated chi-squared statistic differs from that given by chisq.test in R

I'm using R to do a chi-squared test on two sets of data to check if they follow a distribution. In the first case the chi-squared statistic given by chisq.test matches the one calculated by hand, but in the other case it does not.

First case

> e1 = c(18, 38, 78, 107, 107, 72, 35, 15)
> o1 = c(10 ,30 ,80 ,180 ,60 ,40 ,40 ,30)
> sum((o1-e1)**2/e1)
[1] 105.6762
> chisq.test(o1,p=e1,rescale.p=T)$statistic X-squared 105.6762  Both methods give 105.67 as the statistic. But in the other case: > o2=c(401, 1235, 2989, 5682, 8489, 9966, 9196, 6668, 3800, 1701, 599, 165) > e2=c(385, 1204, 2945, 5643, 8472, 9966, 9186, 6635, 3754, 1665, 578, 157) > sum((o2-e2)**2/e2) [1] 5.111822 > chisq.test(o2,p=e2,rescale.p=T)$statistic
X-squared
3.301292


Now the statistics are different and I don't understand why. Am I doing something wrong or missing something?

-
Loosely related: What is wrong with this chi-squared calculation? – gung Apr 15 '14 at 14:56

This isn't really a software issue, as it hinges on a misapplication or misunderstanding of chi-square tests.

1. Your manual calculation in the second case takes no account of the fact that o2 and e2 have different totals, e2 50590, o2 50891.

2. R's result is what I get independently in different software if I scale the expected e2 to sum to the sum of the observed o2.

3. It is surprising in any case that the expected frequencies come as integers. This is unusual in practice. My guess is that this is a two-way problem being presented wrongly as a one-way problem. If so, a chi-square test yields 1.6304 with 11 d.f. and $P =$ 0.999 and the result rings alarm bells as almost too good to be true!

-
Different totals in e2 and o2 were the problem. Thanks! But I don't understand what you mean by this being a two-way problem. I'm comparing two normal distributions and the type of test shouldn't depend on whether the data are integers of floats (though I admit floats for expected frequencies make more sense). – numentar Apr 15 '14 at 12:31
I set aside the claim that you have two "normal" distributions. The main point is that, from what you say, your two vectors of frequencies have the same standing as being both vectors of observed counts. That being so, it is not that one is observed and the other is expected; otherwise put, why not reverse which is observed and which expected? You can separately test whether each is compatible with a normal distribution, or you can jointly test whether they are consistent with the same distribution. Expected means what it says: the expected frequencies are based on some hypothesis. – Nick Cox Apr 15 '14 at 12:42
Chi-square is a lousy way to test for normality, by the way. Use a normal probability plot or (if a $P$-value is something you feel you really need) a dedicated test such as Shapiro-Wilk or Doornik-Hansen. – Nick Cox Apr 15 '14 at 12:47
Thanks for the tips! I'm actually not testing for normality per se. I just generated the data for this post from two normal distributions. But why cannot the expected frequencies be integers? Say I have census data for age groups from an entire country. Now I do random selection 3 years later and want to see if the sample age distribution is different from the population age distribution 3 years ago. Both datasets would originally be counts. – numentar Apr 15 '14 at 13:07
In your last case, the expected frequencies still have to have the same total as the observed frequencies. If your hypothesis is "like this empirical distribution", scaling to the same total means that expected frequencies being integers is unlikely, but admittedly not impossible. You may be spoiled by software doing the scaling for you, but it is needed. – Nick Cox Apr 15 '14 at 13:15