2
$\begingroup$

For my local charity I have to organize random draws, each week to give a prize. I just bought a random machine generator, which randomly spits one ball out of 10.

Before using the machine in public I made some tests in private to ensure that the machine is properly random and I am not accused of cheating.

So I made about 1300 random draws and performed a chisquare test.

I got the following chi square value: 18.13

Since there are 10 possible outcomes, there are 9 degrees of freedom.

Here are the values for 9 df:

ν     0.100  0.050   0.025   0.010   0.005   0.001
9   14.6837 16.9190 19.0228 21.6660 23.5893 27.8772

18.13 is greater that 14.6 and 16.9 does it mean that I am 95% confident that the machine is not random and I should change it? Or is 1300 experiments not enough to make a conclusion?

Improve this question
$\endgroup$
2
$\begingroup$

If the machine is random, you can expect $\frac{1300}{10}=130$ draws out of 1300 for any given ball.  (This is well above the recommended minimum expected value of 5 per category, so you should be fine.)

With the observed frequencies for each ball, calculate the squared Pearson residual: $$R_i^2 = \frac{(O_i-E_i)^2}{E_i}$$ (again, for this problem, $E_i = 130$ for all the categories).

The sum of these (squared) residuals is the chi-squared test statistic $\chi^2=\sum R_i^2$, with 9 degrees of freedom (one less than the number of categories).  Last, you need to calculate the $P$-value.  Using R

pchisq(18.13, 9, lower.tail = TRUE)

(where I used the chi-square test statistic you reported in the question).  I get p=.03.  So, we can assess the null hypothesis.

If the machine is random, the observed frequencies should be very close to the expected frequencies. Thus, the null could be $$H_0 : \text{the machine is random}$$ or $$H_0 : p_1 = p_2 = ··· = p_{10} = \frac{1}{10}$$ If you use a conventional significance level of $\alpha=.05$, then you would reject the null (machine is not random).  If you use $\alpha=.02$, then you would fail to reject the null (no reason to think the machine is not random).

The problem here is that your sample size can actually be too large.

Improve this answer
$\endgroup$
  • $\begingroup$ ok thank you, so the interpretation is purely subjective here? Would you reject or accept the null hypothesis personally? I also dont understand when you say that the sample is too large, isnt it better when we have more data points? $\endgroup$ – user2505650 Apr 22 '18 at 12:01
  • 1
    $\begingroup$ For chi-square goodness-of-fit tests, very large samples always lead to p-values that are less than the significance level...so you are always rejecting the null hypothesis (this is referred to as being over-powered). As for the decision for the hypothesis test, that is entirely your call...you pick the $\alpha$ level you feel comfortable with and go with it. $\endgroup$ – Gregg H Apr 22 '18 at 12:21
  • $\begingroup$ ok thank you, do you recommend any test for this kind of problem? $\endgroup$ – user2505650 Apr 22 '18 at 12:48
  • 1
    $\begingroup$ If I were conducting this type of test, I would run the test with a goal of an expected frequency of ≈15 (ie 150 trials). If I were being extra careful, I would run the same test 100 times to see if about 5 of the tests are flagged as statistically significant. $\endgroup$ – Gregg H Apr 22 '18 at 14:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.