4
$\begingroup$

I have a random variable $X$, and I need to give an example when $X$ and $X^2$ are independent.

I can choose whatever distribution I please, I can say $X$ is a Bernoulli, uniform, or any other distribution.

To prove this, I need to show that $P(X \cap X^2) = P(X)P(X^2)$, but I'm failing to do so.

I tired keeping it simple, taking:

$$X \sim Bernoulli(p)$$

In this case: $$X = X^2$$ and $p$ is probability for succeess and $1-p$ for failure: $$P(X^2) = P(X) = p$$

I also said that $$P(X \cap X^2) = P(X)P(X^2) = P(X)P(X) = p^2$$

But what is $P(X \cap X^2)$? I need to show that it is also $p^2$ in order for this to work. Is my solution enough to prove the rule of independence $P(A \cap B)=P(A)P(B)$ ?

Thank you.

$\endgroup$
9
  • 4
    $\begingroup$ The notation $P(X \cap X^2)$ is inappropriate, as reserved for events, rather than rv's. $\endgroup$
    – Xi'an
    Commented Feb 27, 2023 at 12:34
  • $\begingroup$ In the Bernoulli example, $X=X^2$ so they can hardly be independent, unless $p=0,1$. $\endgroup$
    – Xi'an
    Commented Feb 27, 2023 at 12:35
  • 1
    $\begingroup$ Consider $P(X=1, X^2 =1)$ and $P(X=1) P(X^2 =1)$. Are they equal (for $p \in (0,1)$)? $\endgroup$
    – statmerkur
    Commented Feb 27, 2023 at 12:39
  • 1
    $\begingroup$ You can slightly modify your example by changing the range of r.v. from $\{0 , 1\}$ to $\{-1 , 1\}$. $\endgroup$
    – Zhanxiong
    Commented Feb 27, 2023 at 12:53
  • 1
    $\begingroup$ @Xi'an My point is that the example can be made slightly less trivial (i.e., $X$ itself can be non-degenerate). $\endgroup$
    – Zhanxiong
    Commented Feb 27, 2023 at 14:28

2 Answers 2

3
$\begingroup$

I want to add that a sufficient and necessary condition for $X$ and $X^2$ are independent is that $X^2$ is degenerate.

The sufficiency is trivial. Conversely, suppose $X$ and $X^2$ are independent. It then follows by $X^2$ is a function of $X$ that $X^2$ and $X^2$ are independent, whence \begin{align} E[X^4] = E[X^2 \cdot X^2] = E[X^2]E[X^2], \end{align} i.e., $\operatorname{Var}(X^2) = 0$, which shows that $X^2$ is degenerate.

$\endgroup$
4
  • $\begingroup$ Why does this show $\text{Var}(X^2)=0?$ That seems to require $\mathbb E[X^2]=0), which might be the case but does not have to be. $\endgroup$
    – Dave
    Commented Feb 27, 2023 at 19:00
  • 1
    $\begingroup$ @Dave By definition, $\operatorname{Var}(X^2) = E[X^4] - E[X^2]E[X^2]$. And it has been shown $E[X^4] = E[X^2]E[X^2]$. $\endgroup$
    – Zhanxiong
    Commented Feb 27, 2023 at 19:04
  • $\begingroup$ I think your answer would benefit from making that explicit, rather than being left as an exercise for the reader. $\endgroup$
    – Dave
    Commented Feb 27, 2023 at 19:06
  • $\begingroup$ @Dave Every step is explicitly spelled out. Which part do you think is not "explicit" enough? (I didn't leave any exercise...) $\endgroup$
    – Zhanxiong
    Commented Feb 27, 2023 at 19:09
2
$\begingroup$

A cool answer is that $X$ does not need to be degenerate. Indeed, take $X$ to have the following.

$$ P(X = 1) = p\\ P(X = -1) = 1-p\\ P(X\notin\{-1,1\}) = 0\\ p\in(0,1) $$

Then $P(X^2 = 1)= 1$ and $P(X^2\ne 1)=0$, so, yes, $X^2$ is degenerate.

There are two cases to consider.

$1)$ $P((X=-1)\cap(X^2=1)) \overset{?}{=} P(X=-1)P(X^2=1)$

$2)$ $P((X=1)\cap(X^2=1)) \overset{?}{=} P(X=1)P(X^2=1)$

In the first case, the left side is $(1-p)\times 1=1-p$. The right side is $1-p$, so the equality holds.

In the second case, the left side if $p\times 1=p$. The right side is $p$, so the equality holds.

$\square$

ORIGINAL

To show that $X$ and $X^2$ can be independent, pick $X$ to be a degenerate Bernoulli distribution with all probability on $0$. Then $X^2$ also is a degenerate distribution with all probability on $0$. Since the sample space of each of these variables is $\{0,1\}$, there are four cases to consider.

  1. $P((X=0)\cap(X^2=0)) \overset{?}{=} P(X=0)P(X^2=0)$

  2. $P((X=1)\cap(X^2=0)) \overset{?}{=} P(X=1)P(X^2=0)$

  3. $P((X=0)\cap(X^2=1)) \overset{?}{=} P(X=0)P(X^2=1)$

  4. $P((X=1)\cap(X^2=1)) \overset{?}{=} P(X=1)P(X^2=1)$

In $\#1$, the right side is $1\times 1=1$. The left side refers to the probability of the event of both $X$ and $X^2$ being zero. Since both $X$ and $X^2$ are always zero, $P((X=0)\cap(X^2=0))=1$. Therefore, the equality holds.

In $\#2$, the right side is $0\times 1=0$. The left side refers to the probability that $X=1$ yet $X^2=0$, which is impossible, so $0=0$, and the equality holds.

In $\#3$, the right side if $1\times 0=0$. The left side refers to the probability that $X=0$ yet $X^2=1$, which is impossible, so $0=0$, and the equality holds.

In $\#4$, the right side is $0\times 0=0$. The left side refers to the probability that $X=1$ and $X^2=1$, which is impossible, so $0=0$, and the equality holds.

Since all four equalities hold, for all events in the sample space $\{0,1\}$, the joint probability (the $P(A\cap B)$ in the question) is equal to the product of the marginal probabilities (the $P(A)P(B)$ in the question), and independence holds.

Therefore, at least for this situation with degenerate distributions, $X$ and $X^2$ can be independent. $\square$

But, Dave, $X$ and $X^2$ are perfect predictors of each other! How could they be independent?

That's a reasonable complaint about this, and the justification comes from $X$ and $X^2$ being degenerate distributions with all probability on one value. Yes, knowing the value of either $X$ or $X^2$ allows you to know the value of the other. However, you always know the values of $X$ and $X^2$, since all probability is on one value. Therefore, knowing the value of either $X$ or $X^2$ gives you no additional information about the other, and independence is about using one variable to learn information about the other that you do not get from the marginal distribution of the other. Since the marginal distributions tell the whole story, there is no more information to be gained about the joint probability by knowing a marginal probability.

$\endgroup$
5
  • $\begingroup$ I suspect that at least one of the variables has to be degenerate for this to work, and I would be very interested to read a counterexample. $\endgroup$
    – Dave
    Commented Feb 27, 2023 at 15:57
  • 1
    $\begingroup$ Awesome! The first example you added is a great, very simple and gets the job done. $\endgroup$ Commented Feb 27, 2023 at 16:25
  • $\begingroup$ @ProgrammingNoob Happy to help! Is it clear why $X$ and $X^2$ can be dependent, even if (as my examples show), they do not have to be dependent? $\endgroup$
    – Dave
    Commented Feb 27, 2023 at 16:27
  • 1
    $\begingroup$ Yes, it's a bit tricky, intuitively one might think they must always be dependent, but the equations show the opposite. Cool statistics. $\endgroup$ Commented Feb 27, 2023 at 16:32
  • 1
    $\begingroup$ @Dave Your "suspect" is true. See my answer for the proof. $\endgroup$
    – Zhanxiong
    Commented Feb 27, 2023 at 17:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.