0
$\begingroup$

I can grasp that the LLN is basically saying that deviations grow slower then the total number etc, but what I can't intuitively grasp is “why” 1 standard deviation would actually “be” proportional to square root of N… for example… the square root of 500 is 22.36… if a coin is flipped 500 times then we’d expect heads and tails (not to be exactly 250/250) but rather be ‘off’ by about 22–23 flips or so either side... which is one standard deviation (so the more likely result would be around 227 heads / 273 tails etc)… again, the square root of 1000 is 31.62… so we’d expect heads or tails not to be exactly 500/500 but potentially be ‘off’ by about 32 flips or so either side (1 SD). It's this concept of “when flipping coins their deviations respect/stick/play out/obey/work towards the boundaries of a mathematical standard deviation formula” I’m having difficulty understanding. Why would square rooting the number of physical coin flips (N) be very close to what the deviation in flips would actually be? I'm not a mathematician, I'm a laymen. It's this concept of... "you most likely won't get exactly 250/250 after 500 flips, in reality there's a 68-95% chance you'll be off by about 22/23 flips, which just so happens to be the same result of square rooting the number of flips"... this has been bugging me, as in ‘why’ would it even be the case? Please can someone explain this reason why as simply as possible.

$\endgroup$
  • $\begingroup$ "Proportional to" is not the same as "equal to". As Tim notes below, the standard deviation is $0.5\times\sqrt{n}$, so it IS proportional to (multiplied by a constant) $\sqrt{n}$, but it is not equal to $\sqrt{n}$. $\endgroup$ – dankernler Jul 9 '17 at 4:17
  • $\begingroup$ Thanks. But 'why' does it work? Why would square rooting 'N' reveal the answer to one standard deviation? It's like saying square root the length of track that Usain Bolt runs and it somehow reveals the speed he ran at (I know it doesn't but you get the gist)... why would square rooting the number of trials actually reveal standard deviations (one or two)... that would likely occur in reality? $\endgroup$ – davches Jul 10 '17 at 18:45
  • $\begingroup$ I'm not sure there's an intuitive understanding of the standard deviation of a binomial variable. The mean, yes (multiply the number of trials by the probability of success), but not the standard deviation. $\endgroup$ – dankernler Jul 11 '17 at 2:39
2
$\begingroup$

Where did you get those claims? We are talking here about binomial distribution with probability of success $p=0.5$. Binomial random variable $Y$ is defined in terms of sum of $n$ independent and identically distributed Bernoulli random variables $X_i$, i.e. $Y = \sum_{i=1}^n X_i$. The simplest case is $n=1$, where

$$ E(X) = \Pr(X=0) \times 0 + \Pr(X=1) \times 1 = p \times 0 + (1-p) \times 1 = p $$

Recall that variance is defined as

$$ \mathrm{Var}(X) = E(X^2) - E(X)^2 $$

so to calculate it we need one more component

$$ E(X^2) = \Pr(X=0) \times 0^2 + \Pr(X=1) \times 1^2 = p \times 0 + (1-p) \times 1 = p $$

This leads to

$$ \mathrm{Var}(X) = E(X^2) - E(X)^2 = p - p^2 = p(1-p) $$

If we have $n$ such variables variance becomes $n p(1-p)$. You can easily verify this by simulation (either throwing coins by yourself, or using computer pseudo-random number generator). If this is still unclear, you can refer to some elementary probability handbook.

Standard deviation of binomial distribution with $p=0.5$ is

$$ \sqrt{np(1-p)} = \sqrt{n \times 0.5 \times 0.5} = \sqrt{n} \times 0.5 \ne \sqrt{n} $$

You are also mentioning the normal approximation of binomial distribution and the 68–95–99.7 rule, but with normal distribution 95% of values is within two standard deviations from mean, i.e. $\sqrt{n} \times 0.5 \times 2 = \sqrt{n}$. On another hand 68% of the values lies within one standard deviation from mean, i.e. $\sqrt{n} \times 0.5$. If you want to be talking about 68-95% chance then you would need to define it in terms of interval containing those two values and $\sqrt{n}$ is just the upper bound of it.

Again, you can verify this result on theoretical grounds or via simulation. What you'd learn is that in this case normal approximation works pretty well.

$\endgroup$
  • $\begingroup$ Thanks for your response, it's because my knowledge is limited in this area. I'm trying to achieve the intuitive click where it makes sense. Even at the upper bound, why would performing a mathematical calculation (square rooting the amount of times a coin is flipped (N)) equate to the proportionality constraints of actual physical flip deviations. I'm aware there's no hard fast rule, it just seems odd to me that when physically flipping 500 coins, the deviations between heads/tails will likely be around 23 either way, which just so happens to be the result of square rooting the total flips... $\endgroup$ – davches Jul 8 '17 at 21:56
  • 1
    $\begingroup$ @davches but this is covered in my answer: it directly follows from the standard deviation of binomial distribution. Do you ask about derivation of standard deviation of binomial distribution? $\endgroup$ – Tim Jul 8 '17 at 22:46
  • $\begingroup$ I'm unsure of what that even means, need to look into that. I think what I'm trying to intuitively understand is whatever the calculation is or whatever the term is called, why does it work? $\endgroup$ – davches Jul 9 '17 at 7:37
  • $\begingroup$ You might need a stronger statistical background to understand. Unfortunately, not every concept has an intuitive meaning. $\endgroup$ – dankernler Jul 11 '17 at 2:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.