Given a pseudo-random number generator of standard normal distribution, I generate a matrix $(z_{t,n})_{(t,n)=(1,1)}^{(T,N)}$ of samples. I computed the mean of the cumulative sum, i.e., $\displaystyle s_t = \frac1N\sum_{n=1}^N\sum_{s=1}^tz_{s,n} $. The expected standard deviation of $s_t$ should be $\sigma(s_t)=\sqrt\frac{t}{N}$. However, for $(T,N)=(600,4000)$, I have several $|s_t|$'s exceeding $2.5 $-$3$ times $\sigma(s_t)$. Does this indicate the pseudo-random number generator is bad?
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
3
Your $s_t$ should be normally distributed. If a variable is normally distributed, you expect it to fall more than $2\sigma$ from the mean 4.5% of the time, and more than $3\sigma$ from the mean 0.27% of the time. So having "several" values beyond these bounds doesn't indicate that your RNG is bad.
To be more rigorous, you need to say exactly what fraction are exactly where. The formal way to do this is with a Kolmogorov-Smirnov test, and you should definitely do one if you are suspicious of your generator. In fact, there is a massive battery of tests that are traditionally run on RNGs. See, for example, the dieharder tests.
-
$\begingroup$ Thank you for the answer. I am aware of all you have said except the Dieharder test suite. The Kolmogorov-Smirnov test is best performed on the original sample itself rather than $s_t$. I was wondering if there was anything gained from examining the cumulative sum instead of the sample themselves. I was not clear about it in this question. I actually asked about the normality and independence test in another related question stats.stackexchange.com/q/280610/44368. I would very much like to hear your opinion there. $\endgroup$– HansMay 19, 2017 at 22:47
-
$\begingroup$ Yes, there is something to be gained by doing a KS test on this transformed data. Your n will not be as high as for the original data, but you will be testing for serial correlations (non-independence) that show up as distortions in multi-dimensional distributions constructed from the original distribution. Essentially all the RNG test harnesses work this way, often constructing distributions in 10s-100s of dimensions with known, testable properties. $\endgroup$ May 19, 2017 at 23:04
-
2$\begingroup$ That said, usually test RNG test harnesses are run on new uniform RNGs. Nonuniform RNGs are then constructed from uniform RNGs via established algorithms (Box-Muller, Ziggurat, etc.) If you are using an established uniform RNG, and an established nonuniform transformation algorithm, I would be satisfied with a cursory check of your code (e.g. your original KS) and would be very surprised to find new effects in higher-dimensional tests. $\endgroup$ May 19, 2017 at 23:10