Is the Law of Large Numbers Related to the Occurrence of Rare Events Over Many Trials?

Question

I recently watched an episode of "The Big Bang Theory" where Sheldon makes a comment about the Law of Large Numbers. In the episode, Sheldon realizes he needs eggs, and almost immediately afterwards, Penny knocks on the door and offers to go shopping. Sheldon mentions that "this would be one of those circumstances that people unfamiliar with the law of large numbers would call a coincidence."

This got me thinking: Does the Law of Large Numbers imply that if you run something many times, even rare events are bound to happen?

From what I understand, the Law of Large Numbers is about the average of sample outcomes approaching the expected value or population mean as the number of trials increases. However, Sheldon's comment suggests that it also deals with the inevitability of rare events occurring given enough trials.

Is Sheldon's interpretation of the Law of Large Numbers accurate in a statistical sense? Or is this more akin to the "Gambler's Fallacy" or some other concept?

Answer 1

I agree: large number laws are about stochastic convergence of the sample mean to the population mean. The scene you referenced is talking about occurrence of coincidences/rare events. There is no obvious connection. It's possible that the person who wrote that joke knows nothing more about the Law of Large Numbers than its name, and was hoping the same was true of the audience.

Let's try to be generous though. Can we find some way of twisting a large number law to conclude what Sheldon wants? I assume Sheldon is trying to say something to the effect of "The Law of Large Numbers tells us that I will observe surprising things". Let $X_t$ denote the most recent sense-perception of Sheldon at discrete time $t$. Let $Z_t=\mathbb{1}[X_t=S]$ be a random variable which takes value 1 when something that Sheldon would find very surprising occurs, such as hearing a knock on the door from Penny proffering a shopping trip after he had just expressed a need for eggs, but crucially also other things, such as another friend knocking on his door with shopping on their mind, or a neighbor coming by saying they ran out of room in their fridge and desperately need to offload a carton of eggs onto someone, etc. Otherwise, $Z_t$ will take value $0$.

Let $\bar{Z}_t$ denote the mean of all observed $Z_t$. At least one surprising thing has occurred if $\bar Z_t >0$. Let us denote the long run probability of something surprising happening by $p_\mathcal{S}$, which we'll assume is strictly positive; that is, we have $\mathbb{E}[\bar Z_t]=p_\mathcal{S}$. The weak law of large numbers tells us that $\forall\epsilon$, $\forall\delta$, $\exists M$ such that for $t\geq M$, $P(|\bar Z_t - p_\mathcal{S}| < \epsilon) > 1-\delta$. If we choose $\epsilon = \frac{p_\mathcal{S}}{2}$, we get that $P(|\bar Z_t - p_\mathcal{S}| < \frac{p_\mathcal{S}}{2}) > 1-\delta $, and note that $P(\bar Z_t > \frac{p_\mathcal{S}}{2}) \geq P(|\bar Z_t - p_\mathcal{S}| < \frac{p_\mathcal{S}}{2})$, and, since $p_\mathcal{S}$ was assumed strictly positive, that $P(\bar Z_t > 0) \geq P(\bar Z_t > \frac{p_\mathcal{S}}{2})$, or in other words, that for any $\delta$, for sufficiently large $t$:

$$ P(\bar Z_t > 0)> 1-\delta $$

In other words, $P(\textrm{Something Surprising Happens})$ is arbitrarily close to 1, as Sheldon desired.