# Convergence of random variables

Trying to understand the solution given to this homework problem:

Define random variables $X$ and $Y_n$ where $n=1,2\ldots%$ with probability mass functions:

$$f_X(x)=\begin{cases} \frac{1}{2} &\mbox{if } x = -1 \\ \frac{1}{2} &\mbox{if } x = 1 \\ 0 &\mbox{otherwise} \end{cases} and\; f_{Y_n}(y)=\begin{cases} \frac{1}{2}-\frac{1}{n+1} &\mbox{if } y = -1 \\ \frac{1}{2}+\frac{1}{n+1} &\mbox{if } y = 1 \\ 0 &\mbox{otherwise} \end{cases}$$

Need to show whether $Y_n$ converges to $X$ in probability.

From this I can define the probability space $\Omega=([0,1],U)$ and express the random variables as functions of indicator variables as such:

$X = 1_{\omega > \frac{1}{2}} - 1_{\omega < \frac{1}{2}}$ and $Y_n = 1_{\omega < \frac{1}{2}+\frac{1}{n+1}} - 1_{\omega > \frac{1}{2}+\frac{1}{n+1}}$

And from the definition of convergence in probability, we need find to show that $P\{|Y_n-X|>\epsilon\}$ does or does not converge to zero. Which can be written as:

$P\{|1_{\omega < \frac{1}{2}+\frac{1}{n+1}} - 1_{\omega > \frac{1}{2}+\frac{1}{n+1}} - 1_{\omega > \frac{1}{2}} + 1_{\omega < \frac{1}{2}}| > \epsilon \}\;\;(1)$

Now it's easy to see that $\epsilon < 2$ for this to hold, but the solution given states that:

$P\{|Y_n-X|>\epsilon\} = 1 - \frac{1}{n+1} \;\; (2)$

Thus $Y_n$ does not converge in probability to $X$.

My problem is that I don't see the reasoning between (1) and (2). Can anyone shed some insight into intermediate steps/reasoning required to make this step?

• I'm afraid you lost me at the sample space. How can a sample space--defined as the set of all possible outcomes--be an ordered pair in this situation? What is $U$? Why isn't the sample space simply $\{-1,1\}$--the set of all possible outcomes of either $X$ or any of the $Y_n$? I'm also puzzling over equation (2). If $X$ and $Y_n$ are independent--which seems an implicit assumption--then an easy calculation in the two-way table for $X$ and $Y_n$ gives $\Pr(2\gt |Y_n-X|\gt 0)=1/2$ regardless of $n$.
– whuber
Commented Oct 18, 2012 at 21:40
• Are you sure you reproduced the homework as asked? It seems to me that you should be asked to prove that $(Y_n)$ converges to $X$ in distribution. Which is obvious. Commented Oct 19, 2012 at 5:47
• @whuber In the model of random variables as mappings from a sample space $\Omega$ to the real numbers such that for every Borel set $A$, the pre-image $\{\omega \colon X(\omega) \in A\}$ is a member of $\mathcal F$, the $\sigma$-field, and so $P\{X\in A\} = P\{\omega \colon X(\omega) \in A\}$, it is not possible to take $\Omega = \{+1, -1\}$ and $X$ and $Y_n$ to be identity maps (or their negatives) because the probability measure $P(\cdot)$ cannot assign two different values $1/2$ and $1/2 - 1/(n+1)$ to $\{+1\} \in \mathcal F$. I have posted an answer about independent random variables. Commented Oct 22, 2012 at 15:31
• @Dilip Thank you--that's a useful insight. I had also come to the same conclusion for another reason: one of the simplest models of this situation is to use the rational points in $\Omega=[0,1]\times[0,1]$. Under this model we can find simple forms for $X$ and $Y_n$ and draw nice pictures of their preimages (to carve up the unit square into four rectangles).
– whuber
Commented Oct 22, 2012 at 15:35

You're told that $$P(X=1)=P(X=-1)=1/2 \, ,$$ and $$P(Y_n=1)=\frac{1}{2} + \frac{1}{n+1} \;\;\;, \qquad P(Y_n=-1)=\frac{1}{2} - \frac{1}{n+1} \;\;\;,$$ for $n\geq 1$, and you're asked whether or not $Y_n$ converges to $X$ in probability, which means that $$\lim_{n\to\infty} P(|Y_n-X|\geq \epsilon) = 0 \, , \qquad (*)$$ for every $\epsilon>0$.

I will assume that $X$ is independent of the $Y_n$'s.

It is not the case that $Y_n$ converges in probability to $X$, because $(*)$ does not hold for every $\epsilon>0$.

For instance, if we take $\epsilon=1$, then $$P(|Y_n-X|\geq 1)=P(Y_n=1, X=-1) + P(Y_n=-1,X=1)$$ $$= P(Y_n=1)P(X=-1) + P(Y_n=-1)P(X=1)$$ $$= \left(\frac{1}{2} + \frac{1}{n+1}\right) \cdot \frac{1}{2} + \left(\frac{1}{2} - \frac{1}{n+1}\right) \cdot \frac{1}{2} = \frac{1}{2} \, ,$$ for every $n\geq 1$.

• If $X$ is assumed to be independent of the $Y_n$'s, there never can be convergence in probability, can there? Consider the sequence $Y_n$ where all the $Y_n$ have the same distribution as $X$, not changing with the value of $n$. Then, except for degenerate distributions (a.k.a. constant value with probability $1$), $P\{|Y_n - X| \geq \epsilon\}$ has constant value and so the limit also has the same value. Commented Oct 20, 2012 at 13:23
• Consider this: if $X=0$ with probability one, and $Y_n\sim \mathrm{Bernoulli}(1/n)$, with $X$ independent from the $Y_n$'s, then $Y_n$ converges in probability to $X$.
– Zen
Commented Oct 20, 2012 at 22:55
• Your example is perfectly correct but is the trivial case which I excluded when I wrote "... except for degenerate distributions (a.k.a. a constant value with probability $1$ ..." Do you have an example of convergence in probability to an $X$ which is not a constant with probability $1$ for a sequence of independent $Y_n$ that are also independent of $X$? Note also that, just as events of probability $0$ or $1$ are independent of all other events, your $X = 0$ with probability $1$ is trivially independent of all other random variables. Commented Oct 21, 2012 at 12:04
• Hi, Dilip. I've given the trivial example because the way you wrote your comment I'd understood that the $Y_n$'s should necessarily have a degenerate distribution, which they don't in my trivial example (the target $X$ does). Maybe we can prove, as an exercise, that if the distribution of $X$ is not degenerate and $Y_n$ converges in probability to $X$, whatever the distributions of the $Y_n$'s, then $X$ can't be independent of the $Y_n$'s.
– Zen
Commented Oct 22, 2012 at 0:04
• I think you misunderstood my comment. I will write a separate answer even though it is likely that it will not be read by many others. Commented Oct 22, 2012 at 2:18

Given random variables $X, Y_1, Y_2, \ldots,$ with probability mass functions $$p_X(x)=\begin{cases} \frac{1}{2}, &\text{if}~ x = -1, \\ \frac{1}{2}, &\text{if}~ x = +1 \\ 0 &\text{otherwise,} \end{cases} ~~~~~\text{and}~~~ p_{Y_n}(y)=\begin{cases} \frac{1}{2}-\frac{1}{n+1}, &\text{if}~ y = -1, \\ \frac{1}{2}+\frac{1}{n+1}, &\text{if}~ y = +1, \\ 0 &\text{otherwise,} \end{cases}$$ it is straightforward to show that the sequence $\{Y_n\} = (Y_1, Y_2, \ldots)$, of random variables converges in distribution to $X$ (cf. Xi'an's comment on the question). However, the question of whether $\{Y_n\}$ converges in probability to $X$ depends on our assumptions about the joint probability mass functions $p_{X,Y_n}(x,y)$.

• If we assume, as Zen does, that $X$ is independent of each $Y_n$, then, as Zen shows, $\{Y_n\}$ does not converge in probability to $X$. This is not due to the special circumstance of these random variables being kissing cousins of Bernoulli random variables but holds more generally. If $X$ is nondegenerate random variable, that is $X$ does not have constant value with probability $1$, then no sequence $\{Y_n\}$ of random variables independent of $X$ can converge in probability to $X$. Note that if $X$ is a discrete nondegenerate random variable that is independent of a discrete random variable $Y_n$ then $$P\{Y_n = X\} = \sum_i p_X(u_i)p_{Y_n}(u_i) \leq \sqrt{\sum_i \left(p_X(u_i)\right)^2 \sum_i \left(p_{Y_n}(u_i)\right)^2}$$ Now, $\sum_i \left(p_{Y_n}(u_i)\right)^2$ is just $P\{A = B\}$ where $A$ and $B$ are independent identically distributed discrete random variables with common pmf $p_{Y_n}(\cdot)$, and thus the sum is at most $1$ for all values of $n$. On the other hand, since $X$ is nondegenerate by assumption so that $p_X(u_i) < 1$ for all $i$ while $\sum_i p_X(u_i) = 1$, it follows that $\sum_i \left(p_X(u_i)\right)^2$ cannot equal $1$; it is a constant that is strictly smaller than $1$, say $1-\delta$ for some $\delta > 0$. Hence \begin{align} P\{Y_n = X\} &\leq \sqrt{\sum_i \left(p_X(u_i)\right)^2 \sum_i \left(p_{Y_n}(u_i)\right)^2}\\ &\leq \sqrt{1-\delta}\cdot 1\\ &\Rightarrow~~ \lim_{n\to\infty}P\{Y_n = X\} < 1. \end{align} For continuous random variable $X$ independent of continuous random variable $Y_n$, $P\{Y_n = X\} = 0$. Thus, we have the following result:

If $X$ is a nondegenerate random variable, then no sequence $\{Y_n\}$ of random variables, each of which is independent of $X$, can converge in probability to $X$.

• For $n \geq 3$, if $X$ and $Y_n$ are assumed to be dependent random variables with joint probability mass function \begin{alignat}{4} &p_{X,Y_n}(-1, +1)~& &= \frac{2}{n+1},& \qquad &p_{X,Y_n}(+1, +1)~& &= \frac{1}{2}- \frac{1}{n+1},&\\ &p_{X,Y_n}(-1, -1)~& &= \frac{1}{2} - \frac{2}{n+1},& \qquad &p_{X,Y_n}(+1, -1)~& &= \frac{1}{n+1},&\\ \end{alignat} then it is easy to verify that the marginal probability mass functions are as specified. Also, $$P\{Y_n \neq X\} = \frac{3}{n+1} \to 0 ~ \text{as}~ n \to \infty$$ and so the sequence $\{Y_n\}$ converges in probability to $X$.

• For $n \geq 3$, if $X$ and $Y_n$ are assumed to be dependent random variables with joint probability mass function \begin{alignat}{4} &p_{X,Y_n}(-1, +1)~& &= \frac{1}{2}- \frac{1}{n+1},& \qquad &p_{X,Y_n}(+1, +1)~& &= \frac{2}{n+1},&\\ &p_{X,Y_n}(-1, -1)~& &= \frac{1}{n+1},& \qquad &p_{X,Y_n}(+1, -1)~& &= \frac{1}{2} -\frac{2}{n+1},&\\ \end{alignat} then it is easy to verify that the marginal probability mass functions are as specified. Also, $$P\{Y_n \neq X\} = 1 - \frac{3}{n+1} \to 1 ~ \text{as}~ n \to \infty$$ and so the sequence $\{Y_n\}$ does not converge in probability to $X$: it does, however, converge in probability to $-X$.

• If we assume the joint probability mass function alternates (according as $n$ is odd or even) between the two joint mass functions in the above two bullet point, then we get that $\{Y_n\}$ does not converge in probability at all.

Note to OP: The question you have posed "Does $\{Y_n\}$ converge in probability to $X$?" can be made to have whatever answer you like by choosing the joint distribution of $X$ and $Y_n$ appropriately.

• You can choose each $Y_n$ to be independent of $X$ in which case convergence in probability cannot occur. This is proved in Zen's answer to your question for your particular random variables, and in more generality above.

• You can choose the joint distributions so that $\{Y_n\}$ converges in probability to $X$, as described above. Your own partial answer can be modified to ensure that $\{Y_n\}$ converges to $X$ in probability because the sequence has the far stronger property of converging almost surely.

• You can choose the joint distributions so that $\{Y_n\}$ converges in probability but converges to $-X$, not to $X$, as described above.

• You can choose the joint distributions so that convergence in probability does not occur at all, whether to the specified $X$ or to any other random variable.

• Hi, Dilip. Is your argument correct? For every $\epsilon>0$, you're trying to show that $P(|Y_n-X|<\epsilon)$ can't go to $1$ when you take the limit. Then, you consider a strict upper bound for $P(Y_n=X)$, but $P(|Y_n-X|<\epsilon)\geq P(Y_n=X)$. I'm in a hurry here. I'll take another look at your answer later. Thanks.
– Zen
Commented Oct 22, 2012 at 23:36
• @Zen My argument is not that $P\{Y_n = X\} < 1$ for all $n$ but that for all $n$, $$P\{Y_n = X\} \leq \sqrt{\sum_i \left(p_X(u_i)\right)^2}$$ where the upper bound is a fixed quantity that is strictly less than $1$. Thus, $\lim_{n \to \infty}P\{Y_n = X\}$ cannot be any larger than this upper bound. In your proposed counterexample, $p_n$ can approach $1$ arbitrarily closely whereas $P\{Y_n = X\}$ can only approach $1-\delta$ arbitrarily closely (here $\delta$ is some positive number) Commented Oct 23, 2012 at 2:02
• Be careful: your upper bound does depend on $n$, because $\sqrt{\sum_i p^2_{Y_n}(u_i)}$ does not have to be the same number for every $n$.
– Zen
Commented Oct 23, 2012 at 18:50
• @Zen I do not deny that $\sqrt{\sum_i p^2_{Y_n}(u_i)}$ can vary with $n$, but it does not matter. Since $\sum_i p_{Y_n}(u_i) = 1$ is fixed, isn't it correct to assert that $\sum_i p^2_{Y_n}(u_i) \leq 1$ with equality occurring only if $Y_n$ is degenerate with $p_{Y_n}(u_i) = 1$ for one value of $i$ and $p_{Y_n}(u_j) = 0$ for all $j \neq i$? If so, then $$\sqrt{\sum_i \left(p_X(u_i)\right)^2 \sum_i \left(p_{Y_n}(u_i)\right)^2} \leq \sqrt{\sum_i \left(p_X(u_i)\right)^2}.$$ Commented Oct 23, 2012 at 19:30
• Hi Dilip, thank you for your comment. As I see it, the argument isn't correct. My feeling is that you are assuming that something that holds for every $n$ will hold in the limit. I'll post a new question asking for a proof of this proposition.
– Zen
Commented Oct 23, 2012 at 19:41