5
$\begingroup$

Trying to understand the solution given to this homework problem:

Define random variables $X$ and $Y_n$ where $n=1,2\ldots%$ with probability mass functions:

$$ f_X(x)=\begin{cases} \frac{1}{2} &\mbox{if } x = -1 \\ \frac{1}{2} &\mbox{if } x = 1 \\ 0 &\mbox{otherwise} \end{cases} and\; f_{Y_n}(y)=\begin{cases} \frac{1}{2}-\frac{1}{n+1} &\mbox{if } y = -1 \\ \frac{1}{2}+\frac{1}{n+1} &\mbox{if } y = 1 \\ 0 &\mbox{otherwise} \end{cases} $$

Need to show whether $Y_n$ converges to $X$ in probability.

From this I can define the probability space $\Omega=([0,1],U)$ and express the random variables as functions of indicator variables as such:

$X = 1_{\omega > \frac{1}{2}} - 1_{\omega < \frac{1}{2}}$ and $Y_n = 1_{\omega < \frac{1}{2}+\frac{1}{n+1}} - 1_{\omega > \frac{1}{2}+\frac{1}{n+1}}$

And from the definition of convergence in probability, we need find to show that $P\{|Y_n-X|>\epsilon\}$ does or does not converge to zero. Which can be written as:

$P\{|1_{\omega < \frac{1}{2}+\frac{1}{n+1}} - 1_{\omega > \frac{1}{2}+\frac{1}{n+1}} - 1_{\omega > \frac{1}{2}} + 1_{\omega < \frac{1}{2}}| > \epsilon \}\;\;(1)$

Now it's easy to see that $\epsilon < 2$ for this to hold, but the solution given states that:

$P\{|Y_n-X|>\epsilon\} = 1 - \frac{1}{n+1} \;\; (2)$

Thus $Y_n$ does not converge in probability to $X$.

My problem is that I don't see the reasoning between (1) and (2). Can anyone shed some insight into intermediate steps/reasoning required to make this step?

$\endgroup$
  • 1
    $\begingroup$ I'm afraid you lost me at the sample space. How can a sample space--defined as the set of all possible outcomes--be an ordered pair in this situation? What is $U$? Why isn't the sample space simply $\{-1,1\}$--the set of all possible outcomes of either $X$ or any of the $Y_n$? I'm also puzzling over equation (2). If $X$ and $Y_n$ are independent--which seems an implicit assumption--then an easy calculation in the two-way table for $X$ and $Y_n$ gives $\Pr(2\gt |Y_n-X|\gt 0)=1/2$ regardless of $n$. $\endgroup$ – whuber Oct 18 '12 at 21:40
  • 1
    $\begingroup$ Are you sure you reproduced the homework as asked? It seems to me that you should be asked to prove that $(Y_n)$ converges to $X$ in distribution. Which is obvious. $\endgroup$ – Xi'an Oct 19 '12 at 5:47
  • $\begingroup$ @whuber In the model of random variables as mappings from a sample space $\Omega$ to the real numbers such that for every Borel set $A$, the pre-image $\{\omega \colon X(\omega) \in A\}$ is a member of $\mathcal F$, the $\sigma$-field, and so $P\{X\in A\} = P\{\omega \colon X(\omega) \in A\}$, it is not possible to take $\Omega = \{+1, -1\}$ and $X$ and $Y_n$ to be identity maps (or their negatives) because the probability measure $P(\cdot)$ cannot assign two different values $1/2$ and $1/2 - 1/(n+1)$ to $\{+1\} \in \mathcal F$. I have posted an answer about independent random variables. $\endgroup$ – Dilip Sarwate Oct 22 '12 at 15:31
  • $\begingroup$ @Dilip Thank you--that's a useful insight. I had also come to the same conclusion for another reason: one of the simplest models of this situation is to use the rational points in $\Omega=[0,1]\times[0,1]$. Under this model we can find simple forms for $X$ and $Y_n$ and draw nice pictures of their preimages (to carve up the unit square into four rectangles). $\endgroup$ – whuber Oct 22 '12 at 15:35
3
$\begingroup$

You're told that $$ P(X=1)=P(X=-1)=1/2 \, , $$ and $$ P(Y_n=1)=\frac{1}{2} + \frac{1}{n+1} \;\;\;, \qquad P(Y_n=-1)=\frac{1}{2} - \frac{1}{n+1} \;\;\;, $$ for $n\geq 1$, and you're asked whether or not $Y_n$ converges to $X$ in probability, which means that $$ \lim_{n\to\infty} P(|Y_n-X|\geq \epsilon) = 0 \, , \qquad (*) $$ for every $\epsilon>0$.

I will assume that $X$ is independent of the $Y_n$'s.

It is not the case that $Y_n$ converges in probability to $X$, because $(*)$ does not hold for every $\epsilon>0$.

For instance, if we take $\epsilon=1$, then $$ P(|Y_n-X|\geq 1)=P(Y_n=1, X=-1) + P(Y_n=-1,X=1) $$ $$ = P(Y_n=1)P(X=-1) + P(Y_n=-1)P(X=1) $$ $$ = \left(\frac{1}{2} + \frac{1}{n+1}\right) \cdot \frac{1}{2} + \left(\frac{1}{2} - \frac{1}{n+1}\right) \cdot \frac{1}{2} = \frac{1}{2} \, , $$ for every $n\geq 1$.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ If $X$ is assumed to be independent of the $Y_n$'s, there never can be convergence in probability, can there? Consider the sequence $Y_n$ where all the $Y_n$ have the same distribution as $X$, not changing with the value of $n$. Then, except for degenerate distributions (a.k.a. constant value with probability $1$), $P\{|Y_n - X| \geq \epsilon\}$ has constant value and so the limit also has the same value. $\endgroup$ – Dilip Sarwate Oct 20 '12 at 13:23
  • $\begingroup$ Consider this: if $X=0$ with probability one, and $Y_n\sim \mathrm{Bernoulli}(1/n)$, with $X$ independent from the $Y_n$'s, then $Y_n$ converges in probability to $X$. $\endgroup$ – Zen Oct 20 '12 at 22:55
  • $\begingroup$ Your example is perfectly correct but is the trivial case which I excluded when I wrote "... except for degenerate distributions (a.k.a. a constant value with probability $1$ ..." Do you have an example of convergence in probability to an $X$ which is not a constant with probability $1$ for a sequence of independent $Y_n$ that are also independent of $X$? Note also that, just as events of probability $0$ or $1$ are independent of all other events, your $X = 0$ with probability $1$ is trivially independent of all other random variables. $\endgroup$ – Dilip Sarwate Oct 21 '12 at 12:04
  • $\begingroup$ Hi, Dilip. I've given the trivial example because the way you wrote your comment I'd understood that the $Y_n$'s should necessarily have a degenerate distribution, which they don't in my trivial example (the target $X$ does). Maybe we can prove, as an exercise, that if the distribution of $X$ is not degenerate and $Y_n$ converges in probability to $X$, whatever the distributions of the $Y_n$'s, then $X$ can't be independent of the $Y_n$'s. $\endgroup$ – Zen Oct 22 '12 at 0:04
  • $\begingroup$ I think you misunderstood my comment. I will write a separate answer even though it is likely that it will not be read by many others. $\endgroup$ – Dilip Sarwate Oct 22 '12 at 2:18
2
$\begingroup$

Given random variables $X, Y_1, Y_2, \ldots, $ with probability mass functions $$p_X(x)=\begin{cases} \frac{1}{2}, &\text{if}~ x = -1, \\ \frac{1}{2}, &\text{if}~ x = +1 \\ 0 &\text{otherwise,} \end{cases} ~~~~~\text{and}~~~ p_{Y_n}(y)=\begin{cases} \frac{1}{2}-\frac{1}{n+1}, &\text{if}~ y = -1, \\ \frac{1}{2}+\frac{1}{n+1}, &\text{if}~ y = +1, \\ 0 &\text{otherwise,} \end{cases}$$ it is straightforward to show that the sequence $\{Y_n\} = (Y_1, Y_2, \ldots)$, of random variables converges in distribution to $X$ (cf. Xi'an's comment on the question). However, the question of whether $\{Y_n\}$ converges in probability to $X$ depends on our assumptions about the joint probability mass functions $p_{X,Y_n}(x,y)$.

  • If we assume, as Zen does, that $X$ is independent of each $Y_n$, then, as Zen shows, $\{Y_n\}$ does not converge in probability to $X$. This is not due to the special circumstance of these random variables being kissing cousins of Bernoulli random variables but holds more generally. If $X$ is nondegenerate random variable, that is $X$ does not have constant value with probability $1$, then no sequence $\{Y_n\}$ of random variables independent of $X$ can converge in probability to $X$. Note that if $X$ is a discrete nondegenerate random variable that is independent of a discrete random variable $Y_n$ then $$P\{Y_n = X\} = \sum_i p_X(u_i)p_{Y_n}(u_i) \leq \sqrt{\sum_i \left(p_X(u_i)\right)^2 \sum_i \left(p_{Y_n}(u_i)\right)^2}$$ Now, $\sum_i \left(p_{Y_n}(u_i)\right)^2$ is just $P\{A = B\}$ where $A$ and $B$ are independent identically distributed discrete random variables with common pmf $p_{Y_n}(\cdot)$, and thus the sum is at most $1$ for all values of $n$. On the other hand, since $X$ is nondegenerate by assumption so that $p_X(u_i) < 1$ for all $i$ while $\sum_i p_X(u_i) = 1$, it follows that $\sum_i \left(p_X(u_i)\right)^2$ cannot equal $1$; it is a constant that is strictly smaller than $1$, say $1-\delta$ for some $\delta > 0$. Hence $$\begin{align} P\{Y_n = X\} &\leq \sqrt{\sum_i \left(p_X(u_i)\right)^2 \sum_i \left(p_{Y_n}(u_i)\right)^2}\\ &\leq \sqrt{1-\delta}\cdot 1\\ &\Rightarrow~~ \lim_{n\to\infty}P\{Y_n = X\} < 1. \end{align}$$ For continuous random variable $X$ independent of continuous random variable $Y_n$, $P\{Y_n = X\} = 0$. Thus, we have the following result:

If $X$ is a nondegenerate random variable, then no sequence $\{Y_n\}$ of random variables, each of which is independent of $X$, can converge in probability to $X$.

  • For $n \geq 3$, if $X$ and $Y_n$ are assumed to be dependent random variables with joint probability mass function $$\begin{alignat}{4} &p_{X,Y_n}(-1, +1)~& &= \frac{2}{n+1},& \qquad &p_{X,Y_n}(+1, +1)~& &= \frac{1}{2}- \frac{1}{n+1},&\\ &p_{X,Y_n}(-1, -1)~& &= \frac{1}{2} - \frac{2}{n+1},& \qquad &p_{X,Y_n}(+1, -1)~& &= \frac{1}{n+1},&\\ \end{alignat}$$ then it is easy to verify that the marginal probability mass functions are as specified. Also, $$P\{Y_n \neq X\} = \frac{3}{n+1} \to 0 ~ \text{as}~ n \to \infty$$ and so the sequence $\{Y_n\}$ converges in probability to $X$.

  • For $n \geq 3$, if $X$ and $Y_n$ are assumed to be dependent random variables with joint probability mass function $$\begin{alignat}{4} &p_{X,Y_n}(-1, +1)~& &= \frac{1}{2}- \frac{1}{n+1},& \qquad &p_{X,Y_n}(+1, +1)~& &= \frac{2}{n+1},&\\ &p_{X,Y_n}(-1, -1)~& &= \frac{1}{n+1},& \qquad &p_{X,Y_n}(+1, -1)~& &= \frac{1}{2} -\frac{2}{n+1},&\\ \end{alignat}$$ then it is easy to verify that the marginal probability mass functions are as specified. Also, $$P\{Y_n \neq X\} = 1 - \frac{3}{n+1} \to 1 ~ \text{as}~ n \to \infty$$ and so the sequence $\{Y_n\}$ does not converge in probability to $X$: it does, however, converge in probability to $-X$.

  • If we assume the joint probability mass function alternates (according as $n$ is odd or even) between the two joint mass functions in the above two bullet point, then we get that $\{Y_n\}$ does not converge in probability at all.


Note to OP: The question you have posed "Does $\{Y_n\}$ converge in probability to $X$?" can be made to have whatever answer you like by choosing the joint distribution of $X$ and $Y_n$ appropriately.

  • You can choose each $Y_n$ to be independent of $X$ in which case convergence in probability cannot occur. This is proved in Zen's answer to your question for your particular random variables, and in more generality above.

  • You can choose the joint distributions so that $\{Y_n\}$ converges in probability to $X$, as described above. Your own partial answer can be modified to ensure that $\{Y_n\}$ converges to $X$ in probability because the sequence has the far stronger property of converging almost surely.

  • You can choose the joint distributions so that $\{Y_n\}$ converges in probability but converges to $-X$, not to $X$, as described above.

  • You can choose the joint distributions so that convergence in probability does not occur at all, whether to the specified $X$ or to any other random variable.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Hi, Dilip. Is your argument correct? For every $\epsilon>0$, you're trying to show that $P(|Y_n-X|<\epsilon)$ can't go to $1$ when you take the limit. Then, you consider a strict upper bound for $P(Y_n=X)$, but $P(|Y_n-X|<\epsilon)\geq P(Y_n=X)$. I'm in a hurry here. I'll take another look at your answer later. Thanks. $\endgroup$ – Zen Oct 22 '12 at 23:36
  • $\begingroup$ @Zen My argument is not that $P\{Y_n = X\} < 1$ for all $n$ but that for all $n$, $$P\{Y_n = X\} \leq \sqrt{\sum_i \left(p_X(u_i)\right)^2}$$ where the upper bound is a fixed quantity that is strictly less than $1$. Thus, $\lim_{n \to \infty}P\{Y_n = X\}$ cannot be any larger than this upper bound. In your proposed counterexample, $p_n$ can approach $1$ arbitrarily closely whereas $P\{Y_n = X\}$ can only approach $1-\delta$ arbitrarily closely (here $\delta$ is some positive number) $\endgroup$ – Dilip Sarwate Oct 23 '12 at 2:02
  • $\begingroup$ Be careful: your upper bound does depend on $n$, because $\sqrt{\sum_i p^2_{Y_n}(u_i)}$ does not have to be the same number for every $n$. $\endgroup$ – Zen Oct 23 '12 at 18:50
  • $\begingroup$ @Zen I do not deny that $\sqrt{\sum_i p^2_{Y_n}(u_i)}$ can vary with $n$, but it does not matter. Since $\sum_i p_{Y_n}(u_i) = 1$ is fixed, isn't it correct to assert that $\sum_i p^2_{Y_n}(u_i) \leq 1$ with equality occurring only if $Y_n$ is degenerate with $p_{Y_n}(u_i) = 1$ for one value of $i$ and $p_{Y_n}(u_j) = 0$ for all $j \neq i$? If so, then $$\sqrt{\sum_i \left(p_X(u_i)\right)^2 \sum_i \left(p_{Y_n}(u_i)\right)^2} \leq \sqrt{\sum_i \left(p_X(u_i)\right)^2}.$$ $\endgroup$ – Dilip Sarwate Oct 23 '12 at 19:30
  • $\begingroup$ Hi Dilip, thank you for your comment. As I see it, the argument isn't correct. My feeling is that you are assuming that something that holds for every $n$ will hold in the limit. I'll post a new question asking for a proof of this proposition. $\endgroup$ – Zen Oct 23 '12 at 19:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.