Suppose I want to test whether a die is fair. I toss 60 times with outcome $X = (40,20,0,0,0,0)$ where $i$th coordinate denotes times of $i$th face occurs.Consider Hypothest test($\alpha = 0.05$): $$H_0: p_1 = p_2 = ... = p_6 = 1/6 \\ H_1: p_i \ne 1/6 ,for \ some \ i $$
consider test statistics $\sum_{i=1}^{6}\frac{(x_i - n/6)^2}{n/6}$ which has approximately $\chi^2$ distribution with $5$ degrees of freedom where $x_i$ denotes times of $i$th face occurs.
I think I have to do a transformation to make original sample $X$ has mean $(10,10,10,10,10,10)$,then with this new sample $X_{new}$, do bootstrap,right?

  • 1
    $\begingroup$ What problem do you see with applying the usual chi-squared test exactly as you described? $\endgroup$ – whuber Mar 18 '19 at 19:22

Let's assume more generally that the die you are rolling has $m$ sides. Then the statistic $\chi^2$ has an approximate $\chi^2(m-1)$ distribution when the null hypothesis is true. This approximation will improve as $m$ grows, and for $m=6$, this approximation will turn out to be pretty good (especially for the data you are interested in testing).

If you are worried about "how good" the chi square approximation is you can choose to approximate the null sampling distribution of $\chi^2$ empirically with Monte Carlo. (A proper bootstrap using the data doesn't really make sense in this setting, since the null hypothesis fully specifies the distribution). This MC approach can be performed in R as follows.

n <- 60      #How many trials 
m <- 6       #How many sides to the die
B <- 50000   #How many MC samples
samp_distr <- rep(NA, B)
for(i in 1:B){
  x <- sample(m, n, replace=TRUE)
  samp_distr[i] <- sum((table(x)-n/m)^2)/(n/m)
hist(samp_distr, freq=FALSE, breaks=20)
curve(dchisq(x, m-1), add=TRUE, lwd=2, col='orange')

Check out the fit for $m$ equal to $2$ (coin flip) and $6$. The chi-square approximation is already decent for $m=2$, and for $m=6$ it is really quite good. enter image description here A p-value using the chi-square approximation can be obtained with the command

pchisq(test_stat, m-1, lower.tail=FALSE)

and empirically with the command

mean(samp_distr > test_stat)

As long as B is large enough in the code above, the Monte Carlo method will be "more correct" than the chi-square approximation. The benefit gained from sampling is negligible even for $m=2$, and the chi-square approximation is more than adequate.

| cite | improve this answer | |
  • $\begingroup$ +1. For these data, using the classical chi-squared distribution has better theoretical justification and is more likely to be accurate than the bootstrap, too. $\endgroup$ – whuber Mar 19 '19 at 14:11
  • $\begingroup$ @whuber yes.I also use the classical chi-squared distribution.I want to do bootstrap to compare traditional method.Any hint to do bootstrap for this problem? $\endgroup$ – Spaceship222 Mar 19 '19 at 15:30
  • $\begingroup$ @Spaceship122 This method, while similar to a parametric bootstrap, requires no resampling from the data. I am just simulating draws from the null distribution, hence I hesitate to call this a bootstrap in the usual sense. $\endgroup$ – knrumsey Mar 19 '19 at 16:51
  • $\begingroup$ @whuber Interesting. Can you expand a little bit? Other than Monte Carlo error which can be reduced by doing a large number of simulations, I would have thought the sampling distribution constructed here is exact and therefore better than the chi-square approximation. $\endgroup$ – knrumsey Mar 19 '19 at 16:52
  • $\begingroup$ The bootstrap is an asymptotic procedure. It relies implicitly on the assumption that samples you draw from your data will behave essentially like samples from the population. In the present case, the data are so unbalanced that this is hardly a plausible assumption. I have no doubt bootstrapping--or even any statistical procedure that isn't out-and-out demented--will strongly conclude these data are not independent draws from a uniform distribution. But I would give more credence to what the chi-squared distribution tells me than to what the (easily predictable) bootstrap distribution would. $\endgroup$ – whuber Mar 19 '19 at 16:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.