# What is Epsilon Convergence in Probability?

I understand that the formula for probability of convergence is $P[|X_n − X_\infty| \gt \epsilon ]\to 0$ and I can solve problems using the formula. Can anyone explain it intuitively (like I am a five year old), particularly in regards to what $\epsilon$ is?

• We studied $\varepsilon$ in 9th grade in Analysis class in high school. It was one of the toughest thing to understand for me. I don't think you can explain it normal 5 y.o. Oct 27, 2016 at 17:01
• Not literally to a five you old.... Just as clearly as possible Oct 27, 2016 at 17:02
• I, for one, do not understand this in the context of limits. $P[|X_n − X_\infty| < \epsilon ]\to 0$ would allow one to choose an $\epsilon$ no matter how small such that the difference $P[|X_n − X_\infty|$ is less than that would converge, but $P[|X_n − X_\infty| > \epsilon ]$ would not. So, somebody please explain.
– Carl
Oct 27, 2016 at 17:20

Since we're talking about convergence - specifically, in this case, $X_n$ converging to $X_\infty$ - we want to show that $X_n$ gets really, really, really close to $X_\infty$ as $n$ gets larger and larger.

Think of $\varepsilon$ as any really small positive number; say you think $\varepsilon = 0.01$ is good enough. Then in order to show that $X_n$ is really, really, really close to $X_\infty$, we want to show that $X_n$ falls inside $(X_\infty-0.01,X_\infty+0.01)$ for sufficiently large $n$. (Sufficiently large $n$ just means that there is some $n'$ such that for every $n > n'$, $X_n$ is within plus or minus $0.01$ of $X_\infty$ with probability 1.)

But say that I'm not convinced that $X_n$ converges to $X_\infty$ because $\varepsilon=0.01$ just seems too big for me. So instead, let $\varepsilon = 0.0001$. Then I'm convinced that $X_n$ converges to $X_\infty$ (or that $X_n$ is really, really, really close to $X_\infty$) if we can show that, for sufficiently large $n$, $X_n$ falls inside $(X_\infty-0.0001,X_\infty+0.0001)$.

Suppose you have a lot of friends who pick $\varepsilon$ to be smaller and smaller. The idea behind convergence is that for any $\varepsilon > 0$, no matter how small $\varepsilon$ gets, showing that $X_n$ falls inside $X_\infty\pm\varepsilon$ for sufficiently large $n$ demonstrates that $X_n$ converges to $X_\infty$.

In the most basic terms, $\varepsilon$ is just a small positive number. As it relates to convergence, you want to be able to show that for any $\varepsilon > 0$ (so that all of your infinite friends with different $\varepsilon$ values are convinced), the sequence that converges will, at some point, get within plus or minus $\varepsilon$ of the limit to which you believe the sequence converges. If you cannot show that your sequence falls within $\varepsilon$ of the believed limit for some $\varepsilon$, then the sequence cannot converge to that limit.

• Could you say the smaller $\epsilon$ , would require a bigger $n$ to prove it converges? Oct 27, 2016 at 18:22
• That's generally the case, Matt, but not always true. As a trivial example, let's say your sequence is $\{2,2,2,...\}$ and you want to show that this converges to 2. No matter how small your $\varepsilon$ is, $n=1$ will suffice to prove that it converges. However, it's important to point out that, in order to prove something converges within this context, you must be able to show this for *all* $\varepsilon>0$, no matter how small. It is not sufficient to pick one $\varepsilon$ and say it converges. For example, consider the sequence given by $X_n=sin(n)$ and let $\varepsilon = 10$. Oct 27, 2016 at 18:28

### Sequences of random variables.

Intuition comes from metaphors. The following metaphor, which models random quantities by pulling slips of paper out of a container, captures all the essential mathematical elements while glossing over a technical condition ("measurability") needed to make sense of situations with uncountably many tickets.

Consider a tickets-in-a-box model of a sample space $$\Omega$$: the name of each element $$\omega\in\Omega$$ is written on a slip of paper (a "ticket") which is put into the box. Elements with greater probability are named on more tickets.

A random variable $$X$$ is a consistent way of writing a number on each ticket. "Consistent" means that all the tickets for any particular $$\omega$$ all get the same value of $$X$$, written $$X(\omega)$$.

A sequence of random variables $$X_1, X_2, \ldots, X_n, \ldots$$ therefore can be conceived of as a sequence $$X_1(\omega), X_2(\omega),\ldots$$ written on each ticket (again in a consistent way).

$$X_\infty$$ is another random variable, which is one more number written on each ticket.

### Events and probability.

Let $$\epsilon$$ be any real number. We will say more about it below.

The event $$|X_n - X_\infty| \ge \epsilon$$ describes all the tickets $$\omega\in\Omega$$ for which the values $$X_n(\omega)$$ and $$X_\infty(\omega)$$ differ by $$\epsilon$$ or more. It's a subset of the tickets in the box. These tickets form some proportion of the box: that proportion models their probability, $$\Pr\left(|X_n - X_\infty| \ge \epsilon\right)$$.

### Limits.

Every assertion about a limit is a form of mathematical game. When we write that some sequence has a limit $$L$$, what we mean is we can play a game against a hypothetical opponent (who is doing their best to make us lose) and we will always win. In the limit game, your opponent names some positive number--usually a tiny one--which we will call $$\delta$$. You win if you can remove a finite number of elements from that sequence and show that all the remaining elements are within a distance $$\delta$$ of $$L$$. As in any game, you may calibrate your response to your opponent's move: the elements you remove are allowed to depend on $$\delta$$.

### Limits in probability.

Let's apply the limit game to the assertion $$\Pr\left(|X_n - X_\infty| \ge \epsilon\right)\to 0$$. Because this assertion involves an unspecified quantity $$\epsilon$$, your opponent may also specify its value. That makes the game as difficult as possible for you to win.

So, no matter what values of $$\epsilon$$ and $$\delta \gt 0$$ your opponent specifies, your response will be to cross out some finite number of the random variables $$X_i$$ on the tickets. For every remaining random variable $$X_n$$, let the tickets where $$X_n(\omega)$$ differs from $$X_\infty(\omega)$$ by $$\epsilon$$ or more be the "bad" ones for $$n$$. You win the game provided the proportions of bad tickets are always less than $$\delta$$ (for all the $$X_n$$ that remain).

A little thought reveals the subtlety of this game: the bad tickets for $$n$$ do not have to have any relationship to the bad tickets for $$m$$ (where $$n$$ and $$m$$ designate any of the remaining random variables you didn't cross out). In other words, on any given ticket the values $$X_n(\omega)$$ can bounce all over the place. The limit in probability is a statement about what's written on all the tickets in the box but it is not a statement about what might be written on any individual ticket.

• I am five years old. I have no idea what you are talking about. You lost me at "a sample space Ω". You would have lost my less precocious classmates at the first sentence. Thanks for trying.
– user136526
Oct 27, 2016 at 21:22
• @mickeyf You're welcome. I paid attention to the OP's comment at stats.stackexchange.com/questions/242793/….
– whuber
Oct 27, 2016 at 22:20