# Is chi square test too strict?

Below is the histogram and the model distribution which goodness-of fit that I want to test

Judging by the eye, I would say this is a pretty good fit.

However, when I run the chi square test, it fails. The data for observed count, and expected count is as follows:

0   903 1730    2637
2937    3678    3539    3870
3353    3406    2810    2843
2098    2051    1577    1426
982 873 601 521
385 342 232 197
127 104 57  52
23  16  17  10
7   6

0.00    847.25  1707.92     2512.40
3131.78     3484.96     3600.03     3565.79
3449.83     3272.62     3033.35     2737.29
2402.19     2051.91     1709.04     1390.81
1107.99     865.54  663.95  500.70
371.55  271.49  195.43  138.64
96.95   66.83   45.41   30.42
20.08   13.06   8.36    5.27
3.28    2.00


By calculation, chi square statistics is 284. On the loosest standard, a chi square(34) with 1 - a = 0.95 is 48.6

Since 284 > 48.6, the distribution should be rejected.

Really? Or Did I made any mistake?

I take a rough calculation, for 48.6/34 = 1.3, which is the average of accepting level of variance. However, take an example for the 3rd bin, (2637-2512)^2 = 15625, which is 6 times of the expected frequency, 2512. Is it too strict?

sample size too large?

Are large data sets inappropriate for hypothesis testing?

• My eye says the fit and the data are clearly different. And therein lies the utility of formal hypothesis tests: they help remove the subjective, undocumentable, and irreproducible differences that can arise among multiple opinions. If the test is appropriate for the data, then the result is indisputable. What you should instead be asking is whether the size of the difference it detects is of any concern. – whuber Feb 2 '16 at 14:36
• @whuber, in my case, I think the difference between the model and the histogram is acceptable (I understand that the data retrive process would introduce a lot of variance like this). And It's totally OK to be rejected for some gof test. The problem is that, there is some fit here, and I don't know if there is other good way of measuring it and expressing it. I have R square, but it's too insensitve, while chi square is too sensitive. – cqcn1991 Feb 2 '16 at 15:29
• Sensitivity is a good thing when it comes to measuring fits. Regardless, how you measure the goodness of fit ought to depend on what you're attempting to accomplish. As an example, if it is crucial in an application to model the largest values accurately, then the goodness of fit measurement ought to weight the upper tail of the distribution more heavily than the rest. I hope it's clear that these considerations are independent of, and separate from, whether the differences may be statistically "significant." – whuber Feb 2 '16 at 16:27

• Then is there any other approach that I can turn to? I may try K-S test instead. Or any approach that I can quantify the certainty of goodness? Chi square may be too strict for my case, and I don't know how can I make it looser. – cqcn1991 Feb 2 '16 at 6:56
• @cqcn1991, you need to distinguish between statistical significance and subject-matter significance. See this excellent thread. $\chi^2$ test is fine in what it addresses (statistical significance) and I suspect K-S should not differ from it too much. – Richard Hardy Feb 2 '16 at 7:52