A continuous function of a sequence of random vectors converges in probability to the function of the limit

Proposition: If $\{ X_n \}$ is a sequence of k-dimensional random vectors s.t. $X_n \overset{p}{\to} X$ and if $g: R^k \rightarrow R^m$ is a continuous mapping, then $g(X_n) \overset{p}{\to} g(X)$.

Proof: Let $K$ be a positive real number. Then given any $\epsilon > 0$ we have

I don't understand why $g$ is uniformly continuous and could I have some extra steps/explanations for the inclusion statement that derives from this?

• $g$ is uniformly continuous on $\{x : |x| \le K\}$ because this set is compact and continuity on a compact set implies uniform continuity. May 25, 2015 at 13:40
• I fail to see where this proof goes. It appears to be a "sandwich to zero" proof, which means that the last-last expression on the right-hand-side on the last imaged series of inequalities, that is the sum of four probabilities, should go to zero. The first and fourth component do go to zero per the assumption of convergence in probability of $X_n$, but the second and third? The actual limit $X$ can be anything. What am I missing? May 28, 2015 at 12:08
• @AlecosPapadopoulos I will edit in the rest of the proof. May 28, 2015 at 15:00

This all comes down to the axiomatic and self-evident fact that when $\mathcal{A}$ and $\mathcal{B}$ are events, then $\mathbb{P}(\mathcal{A} \cup \mathcal{B}) \le \mathbb{P}(\mathcal{A}) + \mathbb{P}(\mathcal{B})$. Everything else is just manipulation with sets (which can be visualized with Venn diagrams).

Applying this requires decoding the notation, which in turn means remembering that most of it is shorthand for subsets of some sample space $\Omega$. At the outset, it is convenient to view the ordered pair $\mathrm{Z}_n = (\mathrm{X}_n, \mathrm{X})$ as a single random variable with values in the metric space $M = \mathbb{R}^k\times \mathbb{R}^k$.

I am going to reduce most of this notation to statements about the inverse images of functions from $\Omega$ to $\mathbb{R}$.

First, define

$$g^\prime:M \to \mathbb{R},\ g^\prime((\mathrm{X}, \mathrm{Y})) = g(\mathrm{X}) - g(\mathrm{Y}).$$

The functional composition

$$g^\prime \circ \mathrm{Z}_n:\Omega \to \mathbb{R},\ (g^\prime \circ \mathrm{Z}_n)(\omega) = g^\prime\left(Z_n\left(\omega\right)\right)$$

is a real-valued function on $\Omega$.

Let

$$\mathcal{A}_n = \{|g(\mathrm{X}_n) - g(\mathrm{X})| \gt \varepsilon\} = \{\omega\in\Omega\,:\, |g(\mathrm{X}_n(\omega)) - g(\mathrm{X}(\omega))| \gt \varepsilon\}.$$

This is the inverse image of the complement of the real interval $B(\varepsilon) = [-\varepsilon, \varepsilon]$:

$$\mathcal{A}_n = (g^\prime \circ \mathrm{Z}_n)^{-1}(\mathbb{R} \setminus B(\varepsilon)).$$

It's just a bunch of points in $\Omega$. (In words: $\mathcal{A}_n$ consists of all outcomes where the values of $g$ on $\mathrm{X}$ and $\mathrm{X}_n$ differ too much.)

(To simplify the notation, let's write set complements in $\mathbb{R}$ with overbars, as in

$$\bar B(\varepsilon) = \mathbb{R} \setminus B(\varepsilon)$$

for instance.)

Similarly, let

$$\mathcal{B} = \{\mathrm{X} \le K\} = \{\omega\in\Omega\,:\, \mathrm{X}(\omega) \le K\} = \mathrm{X}^{-1}(B(K))$$

(these are the outcomes where $\mathrm{X}$ is bounded by $K$) and

$$\mathcal{B}_n = \{\mathrm{X}_n \le K\} = \{\omega\in\Omega\,:\, \mathrm{X}_n(\omega) \le K\} = \mathrm{X}_n^{-1}(B(K))$$

(the outcomes where $\mathrm{X}_n$ is bounded by $K$).

Finally, writing

$$d:M\to\mathbb{R},\ d((\mathrm{X}, \mathrm{Y})) = |\mathrm{X}-\mathrm{Y}|,$$

set

$$\mathcal{C}_n = \{|\mathrm{X}_n-\mathrm{X}| \gt \gamma(\varepsilon)\} = \{\omega\in\Omega\,:\,|\mathrm{X}_n(\omega)-\mathrm{X}(\omega)| \gt \gamma(\varepsilon)\} = (d \circ \mathrm{Z}_n)^{-1}(\bar B(\gamma(\varepsilon)))$$

(the outcomes where $\mathrm{X}_n$ differs too much from $\mathrm{X}$).

All four sets are thereby seen to be inverse images of real-valued functions.

Bear in mind the relationship between inverse images of functions and complements. When $f:A\to B$ is any function between sets and $C\subset B$, then

$$A \setminus f^{-1}(C) \subset f^{-1}(B\setminus C).$$

The proof is simple: on the left hand side are all elements $x\in A$ that do not get sent to $C$ by $f$. Since $f$--being a function--is defined on all elements of $A$, any such $x$ must get sent to the complement of $C$ in $B$, QED.

With those mechanical, set-theoretic preliminaries out of the way we can now interpret the text. In the statement after "since $g$ is uniformly continuous ..." we see

$$\{|g(\mathrm{X}_n - g(\mathrm{X})| \gt \varepsilon,\ \mathrm{X}\le K,\ \mathrm{X}_n \le K\} = \mathcal{A}_n \cap \mathcal{B} \cap \mathcal{B}_n$$

and on its right hand side is $\mathcal{C}_n$ itself. In the first step following "Hence" we see such sets as

$$\mathrm{X} \gt K" = \mathrm{X}^{-1} (\bar B(K)) \supset \bar{\mathcal{B}}$$

and

$$\mathrm{X}_n \gt K" = \mathrm{X}_n^{-1} (\bar B(K)) \supset \bar{\mathcal{B}_n}.$$

Their probabilities are summed. This suggests we consider the relationship between

$$\mathcal{A}_n \cap \mathcal{B} \cap \mathcal{B}_n \subseteq \mathcal{C}_n$$

and

$$\mathcal{A}_n \subseteq \mathcal{C}_n \cup \bar{\mathcal{B}} \cup \bar{\mathcal{B}_n}.$$

The clarity of this notation now makes it obvious that the first inclusion implies the second, because any $\omega\in\mathcal{A}_n$ that is not in $\mathcal{B}\cap\mathcal{B}_n$ will certainly lie in

$$\Omega \setminus (\mathcal{B}\cap\mathcal{B}_n) = \bar{\mathcal{B}} \cup \bar{\mathcal{B}_n}.$$

The left of the figure shows a Venn diagram with $\mathcal{A}_n \cap \mathcal{B} \cap \mathcal{B}_n$ in gray while the right of the figure shows $\mathcal{C}_n \cup \bar{\mathcal{B}} \cup \bar{\mathcal{B}_n} = \mathcal{C}_n \cup \left(\Omega \setminus \left(\mathcal{B} \cap \mathcal{B}_n\right)\right)$ in gray. The left gray region clearly is contained in the right gray region.

When we apply probabilities, the first inequality following "Hence" follows immediately because the probability of a union cannot exceed the sum of the probabilities, as noted at the outset.

The next inequality is derived in the same fashion: I leave the drawing of its Venn diagram to the interested reader.

• What did you use to make those great venn diagram pictures? May 28, 2015 at 17:28
• @Matthew Mathematica. It's just a collection of RegionPlot objects overlaid via Show, very quick and dirty. The sets themselves are defined by quadratic inequalities, so that a couple of quick changes in the directions of two of them reliably converts the left graphic into the right one.
– whuber
May 28, 2015 at 17:31

Proposition. If $\{X_n\}_{n\geq 1}$ is a sequence of $k$-dimensional random vectors such that $X_n\overset{\Pr}\to X$, and $g:\mathbb{R}^k\to\mathbb{R}^m$ is continuous, then $g(X_n)\overset{\Pr}\to g(X)$.

Proof. For every $K>0$, if $\|X_n(\omega)\|>K$, then $$\|X_n(\omega)-X(\omega)\|+\|X(\omega)\|\geq\|X_n(\omega)-X(\omega)+X(\omega)\|=\|X_n(\omega)\|>K,$$ yielding that either $\|X_n(\omega)-X(\omega)\|>K/2$ or $\|X(\omega)\|>K/2$. Hence, $$\left\{\omega:\|X_n(\omega)\|>K \right\} \subset \left\{ \omega : \|X_n(\omega)-X(\omega)\|>K/2 \right\} \cup \left\{ \omega : \|X(\omega)\|>K/2 \right\},$$ for every $n\geq 1$. Since $g$ is a continuous function, it is uniformly continuous on the compact set $\{v\in\mathbb{R}^k:\|v\|\leq K\}$. Therefore, for every $\epsilon>0$, there exists a $\delta>0$ such that, for $\|X_n(\omega)\|\leq K$ and $\|X(\omega)\|\leq K$, if $\|X_n(\omega)-X(\omega)\|<\delta$, then $\|g(X_n(\omega))-g(X(\omega))\|<\epsilon$. It follows contrapositively that \begin{align} &\left\{\omega: \|g(X_n(\omega))-g(X(\omega))\|\geq\epsilon, \|X_n(\omega)\|\leq K, \|X(\omega)\|\leq K \right\} \\ &\qquad\subset\left\{\omega: \|X_n(\omega)-X(\omega)\|\geq\delta, \|X_n(\omega)\|\leq K, \|X(\omega)\|\leq K \right\} \\ &\qquad\subset\left\{\omega: \|X_n(\omega)-X(\omega)\|\geq\delta\right\}. \end{align} Moreover, defining $A_n=\{\omega:\|X_n(\omega)\|\leq K\}\cap\{\omega:\|X(\omega)\|\leq K\}$ and using the inclusions above, we have \begin{align} \Pr\!\left\{\omega: \|g(X_n(\omega))-g(X(\omega))\|\geq \epsilon\right\} &= \Pr\!\left(\left\{\omega: \|g(X_n(\omega))-g(X(\omega))\|\geq \epsilon\right\}\cap A_n\right) \\ &+ \Pr\!\left(\left\{\omega: \|g(X_n(\omega))-g(X(\omega))\|\geq \epsilon\right\}\cap A_n^c\right) \\ &\leq \Pr\!\left\{\omega: \|X_n(\omega)-X(\omega)\|\geq\delta \right\} + \Pr(A_n^c) \\ &\leq \Pr\!\left\{\omega: \|X_n(\omega)-X(\omega)\|\geq\delta \right\} \\ &+ \Pr\!\left\{\omega:\|X_n(\omega)\|>K \right\} + \Pr\!\left\{\omega:\|X(\omega)\|>K \right\} \\ &\leq \Pr\!\left\{\omega: \|X_n(\omega)-X(\omega)\|\geq\delta \right\} \\ &+ 2\Pr\!\left\{\omega:\|X(\omega)\|>K/2 \right\} \\ &+ \Pr\!\left\{\omega:\|X_n(\omega)-X(\omega)\|>K/2 \right\}, \end{align} for every $n\geq 1$. Since, by hypothesis, $X_n\overset{\Pr}\to X$, it follows that $$\lim_{n\to\infty} \Pr\!\left\{\omega: \|g(X_n(\omega))-g(X(\omega))\|\geq \epsilon\right\} \leq 2\Pr\!\left\{\omega:\|X(\omega)\|>K/2 \right\}, \quad (*)$$ for every $\epsilon>0$, and every $K>0$ (this is a subtle and crucial logical point!). Because increasing $K$ we can make $2\Pr\!\left\{\omega:\|X(\omega)\|>K/2 \right\}$ arbitrarily small, the inequality $(*)$ holds for every $K>0$ if and only if $$\lim_{n\to\infty} \Pr\!\left\{\omega: \|g(X_n(\omega))-g(X(\omega))\|\geq \epsilon\right\}=0,$$ for every $\epsilon>0$, yielding that $g(X_n)\overset{\Pr}\to g(X)$.$\quad\square$

Note. It's much easier to prove this proposition using the fact that $X_n\overset{\Pr}\to X$ if and only if for any subsequence $\{n_i\}\subset\mathbb{N}$ there is a subsequence $\{n_{i_j}\}\subset\{n_i\}$ such that $X_{n_{i_j}}\to X$ almost surely when $j\to\infty$.

• I want to apply this result for the case $g()=\Phi()$ (the CDF of the standard normal distribution). Do you think that $\Phi$ is a continuous function? Jan 17 at 9:17
• Yes, definitely.
– Zen
Jan 17 at 14:44