# Is the Law of Large Numbers Related to the Occurrence of Rare Events Over Many Trials?

I recently watched an episode of "The Big Bang Theory" where Sheldon makes a comment about the Law of Large Numbers. In the episode, Sheldon realizes he needs eggs, and almost immediately afterwards, Penny knocks on the door and offers to go shopping. Sheldon mentions that "this would be one of those circumstances that people unfamiliar with the law of large numbers would call a coincidence."

This got me thinking: Does the Law of Large Numbers imply that if you run something many times, even rare events are bound to happen?

From what I understand, the Law of Large Numbers is about the average of sample outcomes approaching the expected value or population mean as the number of trials increases. However, Sheldon's comment suggests that it also deals with the inevitability of rare events occurring given enough trials.

Is Sheldon's interpretation of the Law of Large Numbers accurate in a statistical sense? Or is this more akin to the "Gambler's Fallacy" or some other concept?

• British statistician David Hand has written an entire book about this, The Improbability Principle. In it he devotes chapters to eight factors that help clarify the apparently high frequency with which rare events occur. According to novelist Terry Pratchett, "million-to-one chances crop up nine times out of ten." As far as laws of large numbers go, consider (for instance) that any event modeled with a continuous distribution (such as the Normal) is rare indeed, because in the model it has a probability of zero. Thus, the chance that some (unspecified) "rare event" occurs is always 100%.
– whuber
Commented Sep 2, 2023 at 14:42
• I tend to agree with you. The fact that the probability for a "rare event" increases given enough independent trials does not require the law of large numbers. Commented Sep 2, 2023 at 18:24
• Law of large numbers (LLN) is about ergodicity. Gambler's fallacy is different, which is about misjudgement based on averages. Probably what Sheldon experienced in the episode is close to the concept of Birthday Paradox: a coincidence which is surprising is actually would occur statistically more often than we think. Commented Sep 2, 2023 at 22:02

Let's try to be generous though. Can we find some way of twisting a large number law to conclude what Sheldon wants? I assume Sheldon is trying to say something to the effect of "The Law of Large Numbers tells us that I will observe surprising things". Let $$X_t$$ denote the most recent sense-perception of Sheldon at discrete time $$t$$. Let $$Z_t=\mathbb{1}[X_t=S]$$ be a random variable which takes value 1 when something that Sheldon would find very surprising occurs, such as hearing a knock on the door from Penny proffering a shopping trip after he had just expressed a need for eggs, but crucially also other things, such as another friend knocking on his door with shopping on their mind, or a neighbor coming by saying they ran out of room in their fridge and desperately need to offload a carton of eggs onto someone, etc. Otherwise, $$Z_t$$ will take value $$0$$.
Let $$\bar{Z}_t$$ denote the mean of all observed $$Z_t$$. At least one surprising thing has occurred if $$\bar Z_t >0$$. Let us denote the long run probability of something surprising happening by $$p_\mathcal{S}$$, which we'll assume is strictly positive; that is, we have $$\mathbb{E}[\bar Z_t]=p_\mathcal{S}$$. The weak law of large numbers tells us that $$\forall\epsilon$$, $$\forall\delta$$, $$\exists M$$ such that for $$t\geq M$$, $$P(|\bar Z_t - p_\mathcal{S}| < \epsilon) > 1-\delta$$. If we choose $$\epsilon = \frac{p_\mathcal{S}}{2}$$, we get that $$P(|\bar Z_t - p_\mathcal{S}| < \frac{p_\mathcal{S}}{2}) > 1-\delta$$, and note that $$P(\bar Z_t > \frac{p_\mathcal{S}}{2}) \geq P(|\bar Z_t - p_\mathcal{S}| < \frac{p_\mathcal{S}}{2})$$, and, since $$p_\mathcal{S}$$ was assumed strictly positive, that $$P(\bar Z_t > 0) \geq P(\bar Z_t > \frac{p_\mathcal{S}}{2})$$, or in other words, that for any $$\delta$$, for sufficiently large $$t$$:
$$P(\bar Z_t > 0)> 1-\delta$$
In other words, $$P(\textrm{Something Surprising Happens})$$ is arbitrarily close to 1, as Sheldon desired.