I am testing pseudo-random number generators and need to perform a chi-squared test. However, I've encountered some difficulties.

Let's take the following example: I have generated 100 numbers, ranging from 1 to 10. The distribution is as follows:

1: 8

2: 12

3: 9

4: 11

5: 16

6: 6

7: 8

8: 10

9: 13

10: 7

From what I was able to understand, next I should calculate D.

$$D = d1 + d2 + d3 + ... + d10.$$

$di =$ square of the difference between the expected value and the observer value, everything over the expected value

$$d1 = ((8 - 10)^2)/10 = 4/10$$

$$d2 = ((12 - 10)^2)/10 = 4/10$$

. . .

$$d10 = ((7 - 10)^2)/10 = 9/10$$

Adding them up results in 84/10 or 8.4.

The next step is comparing this to $X^2$.

That is $X^2[1-\alpha,k-1]$. It is clear that $k=10$. But what value should I use for $\alpha$? And how to I know the value of $X^2$ after I decide what $\alpha$ I am going to use?

It feels that I am close but I just can't figure it out. Many thanks.

  • $\begingroup$ There's really no need to worry what the critical value is when your test statistic is smaller than the df; it's not going to be significant at any reasonable significance level for any integer df. $\endgroup$ – Glen_b Aug 13 '15 at 14:09
  • $\begingroup$ @Glen_b, my impression was the OP wants to understand the general principle and is not so much interested about the specifics of this sample. $\endgroup$ – Christoph Hanck Aug 13 '15 at 14:36
  • $\begingroup$ @ChristophHanck which is why it's a comment (but the point itself is a general principle that applies quite widely) $\endgroup$ – Glen_b Aug 14 '15 at 0:14
  • $\begingroup$ @Glen_b, let us hope that your shortcut (if I may call it like that) is not interpreted as a general feature of hypothesis tests - for example, for t-distributed tests, there is of course no relationship between teststat < d.f. and non-rejection. $\endgroup$ – Christoph Hanck Aug 14 '15 at 5:01
  • $\begingroup$ @Christoph Yes, I should have been clear that it was in relation to the chi-square distributions. $\endgroup$ – Glen_b Aug 14 '15 at 11:42

You are looking for the $1-\alpha$ quantile of the $\chi^2$ distribution with $k-1$ degrees of freedom, the critical value of your test. That is, the value that your test statistic needs to exceed in order for you to reject the null hypothesis of your test.

So the choice of $\alpha$ relates to the significance level at which you test. Typically, $\alpha=.05$, $0.1$ or $0.01$.

These critical values can be computed in for example R via

> qchisq(.95,9)
[1] 16.91898
  • 1
    $\begingroup$ I was originally going to answer along those lines. However, when you are testing random number generators, you are more concerned about the type II error, so conventional wisdom has it to use a larger alpha to compensate, e.g., alpha = 0.2 or 0.25. (The computations are of course similar, and there are tables readily available.) $\endgroup$ – user3697176 Aug 13 '15 at 14:31
  • $\begingroup$ Interesting - did not know that test in particular, so my answer was a rather generic hypothesis testing one! $\endgroup$ – Christoph Hanck Aug 13 '15 at 14:32
  • $\begingroup$ I am having a hard time understanding exactly what alpha and significance level mean. So in my test, the null hypothesis is that the sequence is random. The significance level is the probability of the sequence being random and failing the test, right? So if my test statistic is smaller than 16.91898 (which it is - it's 8.4) - does this mean that there is a 95 chance that the sequence is random? Moreover, what if I had 100 numbers instead of just 10? Would I have 99 degrees of freedom? $\endgroup$ – letsplay14me Aug 14 '15 at 7:06
  • $\begingroup$ Let's say I have 100 numbers and the significance level I want is 0.01. Would I have to compare my test statistic with 2.365? And then if it's smaller than 2.365, then may I conclude that the distribution is random with a confidence level of 99.9%? Is that how this works? $\endgroup$ – letsplay14me Aug 14 '15 at 7:09
  • $\begingroup$ Apologies for this many questions here, but I've also used P-value when implementing some randomness tests based on their description (although I didn't understand everything). For this example (100 numbers, 0.01 significance level) might I compute the P-value and check if it's larger than 0.01? How might I do that? I used a online calculator to compute the P-value for a test statistic of 8.4 and 9 degrees of freedom, and I got 0.49. This is larger than 0.01, but it failed the test right? I thought it was supposed to be smaller than 0.01 to fail the test. Thank you. $\endgroup$ – letsplay14me Aug 14 '15 at 7:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.