Proof of Convergence in Probability

Theorem

Suppose $X_n \stackrel{P}\to X\quad \text{and}\quad Y_n \stackrel{P}\to Y$. Then $X_n+Y_n \stackrel{P}\to X+Y$.

Proof

Let $\epsilon >0$ be given. Using the triangle inequality, we can write $$| X_n-X|+|Y_n-Y| \geq |\left( X_n+Y_n \right) -\left( X+Y \right) | \geq \epsilon$$

Since $P$ is monotove relative to set containment, we have

\begin{align} &P \left[| \left(X_n+Y_n \right)- \left(X+Y \right)| \geq \epsilon \right] \\ & \\ &\leq P \left[|X_n-X|+|Y_n-Y| \geq \epsilon \right] \\ & \\ &\leq P \left[|X_n-X| \geq \epsilon /2 \right]+P \left[|Y_n-Y| \geq \epsilon/2 \right] \end{align}

Then by the hypothesis of the theorem the last two terms go to $0$ which gives us the desired result.

First of all sorry for the messy LaTeX notation, I did not know how to align those two inequalities in order for the second to appear right below the first, although I tried. My questions are what exactly does "P is monotone relative to set containment" mean and also how is the last inequality derived? Thank you.

The meaning of $P$ is monotone relative to set containment is this:

If $A \subseteq B$ then $P(A)\leq P(B)$.

Now here for every $\omega\in\Omega$, we have \begin{align*} A &= \left\{\omega : \left| (X_n(\omega) + Y_n(\omega)) - ( X(\omega) + Y(\omega) ) \right| \geq \epsilon \right\}, \text{ and}\\ B &= \{ \omega : \left| X_n(\omega) - X(\omega) \right| + \left| Y_n(\omega) - Y(\omega) \right| \geq \epsilon \}. \end{align*}

From your first inequality and for every $\omega\in\Omega$ and $\varepsilon >0$, we have

$$\left\{ \left| (X_n(\omega) + Y_n(\omega) - (X(\omega) + Y(\omega)) \right| \geq \varepsilon \right\} \subseteq \left\{ \left|X_n(\omega) - X(\omega)\right| + \left|Y_n(\omega) - Y(\omega)\right| \geq \varepsilon \right\}.$$

So from the statement above

$$P\left( \left| (X_n(\omega) + Y_n(\omega) - (X(\omega) + Y(\omega)) \right| \geq \varepsilon \right) \leq P\left( \left|X_n(\omega) - X(\omega)\right| + \left|Y_n(\omega) - Y(\omega)\right| \geq \varepsilon \right).$$

To show the last inequality, first we prove that for every for every $\omega\in\Omega$ and $\varepsilon >0$ we have

$$\begin{multline*} \{\omega : \left|X_n(\omega) - X(\omega)\right| + \left|Y_n(\omega) - Y(\omega)\right| \geq \varepsilon \}\\ \subseteq \{\omega : \left|X_n(\omega) - X(\omega) \right| \geq \varepsilon/2\} \cup \{\omega : \left|Y_n(\omega) - Y(\omega) \right| \geq \varepsilon/2\}. \end{multline*}$$

This can be simply shown by contradiction, i.e. if

$$\{ \omega : \left|X_n(\omega) - X(\omega) \right| < \varepsilon/2 \} \text{ and } \{\omega : \left|Y_n(\omega) - Y(\omega) \right| < \varepsilon/2 \}$$

then we can sum them up to get $\{ \omega : \left|X_n(\omega) - X(\omega)\right| + \left|Y_n(\omega) - Y(\omega)\right| < \varepsilon \}$, which is a contradiction.

Hence,
\begin{align*} P\bigl( \{ \omega : &\left| X_n(\omega)-X(\omega) \right| + \left| Y_n(\omega) - Y(\omega) \right|\geq \varepsilon \} \bigr)\\ &\leq P\left( \{\omega : \left| X_n(\omega) - X(\omega) \right| \geq \varepsilon/2\} \cup \{\omega : \left| Y_n(\omega) - Y(\omega) \right| \geq \varepsilon/2 \} \right)\\ &\leq P\left( \{\omega : \left| X_n(\omega) - X(\omega) \right| \geq \varepsilon/2\}\right) + P\left( \{\omega : \left| Y_n(\omega) - Y(\omega) \right| \geq \varepsilon/2 \} \right), \end{align*}
where the last inequality comes from $P(C \cup D) \leq P(C) + P(D)$. Cheers!

• Thank you but could you also tell me how the last inequality is derived as well? I do not understand the $\epsilon/2$ inequalities. – JohnK Nov 10 '13 at 21:44
• Sure I can. But let me give you a hint maybe you can figure it out yourself first. Can you prove something like this: if $|C|+|D|\geq \epsilon$ then either $|C|\geq \epsilon/2$ or $|D|\geq \epsilon/2$. As for the 2nd hint: prove it by contradiction ;) – Stat Nov 10 '13 at 22:00
• Well it seems trivial but I cannot prove it, I am just a beginner you see ;) – JohnK Nov 10 '13 at 22:06
• OK, I added some lines to complete the proof. – Stat Nov 10 '13 at 22:31
• (+1) I would just add that the argument for $x+y>\epsilon\Rightarrow (x>\epsilon/2 \;\mathrm{or}\; y>\epsilon/2)$ is more specifically named an argument by contraposition (see en.wikipedia.org/wiki/Contraposition). – Zen Nov 10 '13 at 23:15

If you're aware of an equivalent characterization of convergence in probability, the proof is really short. It is known that $X_n \rightarrow X$ in probability if and only if, for any subsequence $\left \{a_1, a_2, \ldots \right\}$, there exists a sub-subsequence $\left\{a'_1,a'_2,\ldots \right \}$ such that $X_{a'_n} \rightarrow X$ almost surely. This can be proven using the Borel-Cantelli Lemma.

Then, to show that $X_n + Y_n \rightarrow X + Y$ in probability, let $\left\{a_1,a_2,\ldots \right\}$ be an arbitrary subsequence. By taking two successive subsequences of the $\left\{a_n\right\}$, we find a sub-sub-subsequence $\left\{a''_n\right\}$ such that $X_{a''_n} \rightarrow X$ and $Y_{a''_n} \rightarrow Y$, both almost surely. From basic real analysis, $X_{a''_n} + Y_{a''_n} \rightarrow X + Y$ almost surely. By the characterization, we know that $X_n + Y_n \rightarrow X + Y$ in probability.

• Thank you but I am too much of a beginner to know that!Someday I hope! – JohnK Nov 11 '13 at 21:55