Hypothesis testing: Multiple Coin toss trials So, I know how to calculate the probability that (say for number of coins=1000, what probability I'd expect to get 520 heads (say):
(1/2)^520 * (1/2)^480 * (1000!/(520!480!))

and obviously to get the probability that it's >= 520, you'd just sum them up
Sum[(1/2)^h * (1/2)^(1000-h) * (1000!/(h!(1000-h)!)),{h, 520, 1000}]

But, say I ran this trial 20 times. And, the percentage of heads was say
[0.497, 0.475, 0.503, 0.475, 0.505, 0.496, 0.511, 0.498, 0.506, 0.487, 0.524, 0.515, 0.497, 0.505, 0.526, 0.488, 0.498, 0.472, 0.487, 0.496]

So, the mean of those values is 0.49805 and the standard deviation is 0.014476.
It would seem that a fair coin should have a mean of 0.5, and a standard deviation of (Sqrt[n p q]/n) = 0.015811..
I guess .. How would one quantify how "different" my 20 trial set was from what I'd expect the "ideal" coin to be?
Thanks!
 A: Each sequence of 1000 tosses of a fair coin will have its own peculiarities. There are some easily derived statements about what you can expect most of the time.
Some involve counting the number of Heads. Let $X$ be the number of heads in $n = 1000$ independent tosses of a fair coin. Then $\mu=E(X) = 500,\, \sigma^2=Var(X) = 250,\,$ $\sigma = SD(X) = \sqrt{250} = 15.8114.$ So you would expect [exact computation in R]
$$P(\mu-2\sigma \le X\le \mu+2\sigma) \approx 0.95 \approx P(458 \le X \le 532) = 0.9765.$$
diff(pbinom(c(457,532), 1000, .5))
[1] 0.9765306

However, you could get $X = 500$ exactly, which is the most common outcome and yet
have 250 Heads followed by 250 Tails, which would be very 'unusual' because there are only two runs. (A run is a sequence of repeated outcomes.) On average, one would expect about 501 runs in a sequence of 1000 tosses. The fewest possible runs is 2 and the greatest is 1000 (alternating H and T throughout).
Thus you can get a very 'ordinary' value of $X$ and a (tiny or huge) number of runs that makes you wonder whether coin tosses are truly independent.
In order to say whether your 20 consecutive coin tosses are "different from ideal," you would have to state your criterion
for ideal. Do you mean an almost average number of heads, an almost average number of runs, or something else?
You have raised an important question. People who seek algorithms for generating pseudo-random numbers by computer, have long lists of criteria for "satisfactory" behavior.
The idea is to generate numbers by computer that cannot be
distinguished from "random" for practical purposes.
