I'm a bit confused by the definition of convergence of a sequence of random variables. I'm reading Wasserman's All of Statistics and at the start of Chapter 5, he writes:

"Suppose that $X_1, X_2...$ is a sequence of random variables which are independent and suppose each has a N(0, 1) distribution. Since these all have the same distribution, we are tempted to say that $X_n$ converges to $X$~$N(0,1)$. But this can't quite be right since $P(X_n = X) = 0$ for all $n$. (Two continuous random variables are equal with probability 0.)"

Why is it the case that two continuous random variables are equal with probability 0? It seems that he's using a different definition of "equal" as I would have thought two random variables with the same distribution are equal in some sense...

Any help clearing this up is much appreciated!

  • 3
    $\begingroup$ Title of question is kind of misleading, something like "Difference between equality in distribution and equality in value" would be better $\endgroup$ Aug 27 '15 at 18:37

I believe you are mixing up the distributions being equivalent and the value of the random variables being equal.

To show with (maybe) an easier example, consider flipping two coins (i.e. two independent Bernoulli distributions with $p = 0.5$). These two random variables have equivalent distributions. Note that there is no probability statement involved there, it's just a fact. However, if you flip two coins, the probability that the outcome will be the same for both coin flips will be 0.5: if you get heads on the first flip, you have 0.5 probability of getting heads on the second, and if you get tails on the first flip, you again have 0.5 probability of getting tails on the second.

Now, if your random variables are continuous rather than Bernoulli, we already know that for a continuous random variable $X$, $P(X = a) = 0$ for all $a$. Therefore, if in our first draw of a continuous random variable we get $X_1 = x_1$, then the $P(X_2 = x_1) = 0$ (assuming $X_1$ and $X_2$ are independent as stated above).

  • 2
    $\begingroup$ As with @Aksakal's answer, this skirts around the issue, i.e. your answer doesn't break if I define $X_2$ to be equal to $X_1$! The long answer is that if $X_1$ and $X_2$ are independent and continuous, then the joint distribution for $(X_1,X_2)$ is continuous, i.e. has a density in $\mathbb{R}^2$ with respect to Lebesgue measure. Now use the fact that the line $\{ (x_1,x_2) \in \mathbb{R}^2: x_1 = x_2\}$ has zero Lebesgue measure to deduce that $\mathbb{P}(X_1 = X_2) = 0$. $\endgroup$ Aug 27 '15 at 18:42
  • $\begingroup$ It amounts to evaluating $$\int_x \mathbb{P}(X_1 = X_2 \in dx)$$ (hopefully self-explanatory notation) $\endgroup$ Aug 27 '15 at 18:44
  • $\begingroup$ If $X_1$ = $X_2$, this defies the OP's assumption that $X_1$ and $X_2$ are independent with variance of 1. In addition, I don't think that this technical issue is the problem the OP was stuck on. $\endgroup$
    – Cliff AB
    Aug 27 '15 at 18:52
  • $\begingroup$ I'm not disputing that the OP assumes that $X_1$ and $X_2$ are independent, and I agree they're not stuck on a technicality. I'm merely pointing out that independence is a strong assumption, and your explanation doesn't use it transparently. On the other hand, $\mathbb{P}(X_1 = X_2) = 0$ is clearly false in general, even if we insist that $\mathbb{P}(X_i = x) = 0$ for any given value of $x$ ($i = 1,2$). Thus, somehow the explanation doesn't capture what is going on. $\endgroup$ Aug 27 '15 at 18:56
  • $\begingroup$ fair enough. Added the independence assumption. $\endgroup$
    – Cliff AB
    Aug 27 '15 at 19:07

Yes, two random variable from the same distribution are equal in some sense. That sense is "in distribution". Your text is referring to exactly that. He says that the sequence $X_1,X_2\dots$ converges to $X$ in distribution. That's pretty much a definition of this kind of convergence.

If you know what is a probability density function $f(x)$ and can integrate, then the easiest way to understand why the probability of $X_n=X$ is zero is to go back to the definitions: $$P(a<X_n\le a+\delta)=\int_{a}^{a+\delta}f(x)dx$$

Now, when you shrink $\delta\to 0$ the probability will go to zero: $$P(X_n=a)=f(a)*0=0$$. Imagine that you're slicing the bread into thinner and thinner slices. The weight of the slices is getting lighter and lighter. So, the probability that $X_n$ would be equal to any given number $a$ is zero.

  • $\begingroup$ The notation used here is terrible. E.g. the left hand side of the first equation is a deterministic (fixed) number. The right hand side is a random variable! $\endgroup$ Aug 27 '15 at 18:20
  • $\begingroup$ The way you've written it over simplifies the things. For example, how have you used independence between $X_n$ and $X$? If $X_n$ is defined equal to $X$ then how how does your equation break? $\endgroup$ Aug 27 '15 at 18:22
  • $\begingroup$ @P.Windridge, he has the text which describes it all in strict math language. $\endgroup$
    – Aksakal
    Aug 27 '15 at 18:24
  • $\begingroup$ You missed the point, which is that your current equation has [something fixed] = [something random], which is clearly incorrect. $\endgroup$ Aug 27 '15 at 19:00
  • $\begingroup$ @P.Windridge, I'll change the equation, but you're nit picking here $\endgroup$
    – Aksakal
    Aug 27 '15 at 19:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.