Hypothesis testing: Multiple Coin toss trials

So, I know how to calculate the probability that (say for number of coins=1000, what probability I'd expect to get 520 heads (say):

(1/2)^520 * (1/2)^480 * (1000!/(520!480!))


and obviously to get the probability that it's >= 520, you'd just sum them up

Sum[(1/2)^h * (1/2)^(1000-h) * (1000!/(h!(1000-h)!)),{h, 520, 1000}]


But, say I ran this trial 20 times. And, the percentage of heads was say

[0.497, 0.475, 0.503, 0.475, 0.505, 0.496, 0.511, 0.498, 0.506, 0.487, 0.524, 0.515, 0.497, 0.505, 0.526, 0.488, 0.498, 0.472, 0.487, 0.496]


So, the mean of those values is 0.49805 and the standard deviation is 0.014476.

It would seem that a fair coin should have a mean of 0.5, and a standard deviation of (Sqrt[n p q]/n) = 0.015811..

I guess .. How would one quantify how "different" my 20 trial set was from what I'd expect the "ideal" coin to be?

Thanks!

Each sequence of 1000 tosses of a fair coin will have its own peculiarities. There are some easily derived statements about what you can expect most of the time.

Some involve counting the number of Heads. Let $$X$$ be the number of heads in $$n = 1000$$ independent tosses of a fair coin. Then $$\mu=E(X) = 500,\, \sigma^2=Var(X) = 250,\,$$ $$\sigma = SD(X) = \sqrt{250} = 15.8114.$$ So you would expect [exact computation in R] $$P(\mu-2\sigma \le X\le \mu+2\sigma) \approx 0.95 \approx P(458 \le X \le 532) = 0.9765.$$

diff(pbinom(c(457,532), 1000, .5))
[1] 0.9765306


However, you could get $$X = 500$$ exactly, which is the most common outcome and yet have 250 Heads followed by 250 Tails, which would be very 'unusual' because there are only two runs. (A run is a sequence of repeated outcomes.) On average, one would expect about 501 runs in a sequence of 1000 tosses. The fewest possible runs is 2 and the greatest is 1000 (alternating H and T throughout).

Thus you can get a very 'ordinary' value of $$X$$ and a (tiny or huge) number of runs that makes you wonder whether coin tosses are truly independent.

In order to say whether your 20 consecutive coin tosses are "different from ideal," you would have to state your criterion for ideal. Do you mean an almost average number of heads, an almost average number of runs, or something else?

You have raised an important question. People who seek algorithms for generating pseudo-random numbers by computer, have long lists of criteria for "satisfactory" behavior. The idea is to generate numbers by computer that cannot be distinguished from "random" for practical purposes.

• Thanks @BruceET for your answer, and I hear you re: HHHHHTTTTT being a very unusual result for ten tosses of a coin.. But, I was more sort of after just ... given that I ran it for 20 trials, and I got these results, how can I determine how "ordinary" or "unusual" those means are... I think I'm looking for comparing 2 means and 2 standard deviations... But, I really just have 1 mean and 1 standard deviation that I'm comparing against (in this case) 0.5 and 0.015811 (my predicted mean and standard deviation)... Does that make sense?
– bnsh
Sep 9 '20 at 21:52
• Maybe like if the data were this: [0.146, 0.143, 0.146, 0.155, 0.151, 0.157, 0.136, 0.147, 0.145, 0.15, 0.149, 0.144, 0.153, 0.165, 0.164, 0.14, 0.149, 0.159, 0.156, 0.127] (mean: 0.1491 std: 0.0089), how would I quantitatively say that this is "more" unusual than (mean: 0.49805, std: 0.014476)?
– bnsh
Sep 9 '20 at 21:56
• If you know the type of distribution (binomial, Poisson, normal, uniform, exponential, etc.) from which data are sampled and you have a specific doubt (whether there the mean is too far off, the SD is too far off, the number of runs is too far off, etc.), then it is usually possible to find a test (or to do a simulation) that helps you judge just 'how far off' a particular observed value may be. // But if you are just 'worried in general' that 'something' may be wrong, then that's too vague for a quantitative response. Sep 9 '20 at 21:59
• For data in last Comment: More unusual with regard to what? Mean and SD of those 20 observations from whatever dist'n do not seem consistent with mean 0.49804, nor with SD 0.0145, which must come from a different dist'n. The 20 observations seem to come from normal dist'n with $\mu$ in $(0.145, 0.153)$ and $\sigma$ in $(0.007, 0.013).$ // Maybe sample mean 0.498 and SD 0.0145 quite "usual" for some other dist'n. Sep 10 '20 at 2:10