# Rules to apply Monte Carlo simulation of p-values for chi-squared test

I'd like to understand the use of Monte Carlo simulation in the chisq.test() function in R.

I have a qualitative variable which has 128 levels / classes. My sample size is 26 (I was not able to sample more "individuals"). So obviously, I will have some levels with 0 "individuals". But the fact is that I have only a very small number of classes represented out of the 127 possible. As I have heard that to apply chi-squared test we should have at least 5 individuals in each level (I do not completely understand the reason for that), I thought I had to use the simulate.p.value option to use Monte Carlo simulation to estimate the distribution and compute a p-value. Without Monte Carlo simulation, R gives me a p-value < 1e-16. With Monte Carlo simulation, it gives me a p-value at 4e-5.

I tried to compute the p-value with a vector of 26 ones and 101 zeros, and with Monte-Carlo simulation, I get a p-value at 1.

Is it OK to state that, even if my sample size is small compared with the number of possible classes, the observed distribution is such that it is very unlikely that all possible classes exist at the same probability (1/127) in the real population?

• If your data really are that you observed 26 distinct classes out of a sample of 26, then you have essentially no evidence against the hypothesis that all 127 classes have equal probability. This can be assessed with a multinomial distribution calculation. – whuber Jun 27 '13 at 13:49
• "As I have heard that to apply chi-squared test we should have at least 5 individuals in each level (I do not completely understand the reason for that)" -- not quite. The original advice was that the expected count, not the actual count should be at least 5. The aim with that (now long outdated) rule was to try to make sure the chi-square distribution is a reasonable approximation to the discrete distribution of the test statistic. Advice across a slew of papers over the last 4 decades or so is 'that rule is somewhat too strict'. – Glen_b Aug 26 '13 at 23:58