4
$\begingroup$

Suppose we have two continuous random variables $X \sim U(-1,1)$ and $Y=X^2$. I don't understand why they are dependent.

$$E[X] = 0$$ $$E[Y] = \int_{-1}^{1} x^2 dx = 2/3$$ $$E[XY] = \int_{-1}^{1} x^3 dx = 0$$

They are independent to me because $E[XY]=E[X]E[Y]$, so why they are dependent?

$\endgroup$
12
  • 3
    $\begingroup$ Plotting simulated values of $X$ and $Y$ will be enlightening, but you can also work out the density of $Y$ via a change of variables from $X$ in order to compare the probabilities in terms of definition of statistical independence. $\endgroup$
    – Galen
    Commented Feb 4, 2023 at 4:15
  • 2
    $\begingroup$ I find it challenging to find any function $h$ for which $X$ and $h(X)$ are independent. $\endgroup$
    – whuber
    Commented Feb 4, 2023 at 15:55
  • 2
    $\begingroup$ @whuber: one such function $h$ is $h(x)=0$. Another one is $h(x)=5$. $\endgroup$ Commented Feb 4, 2023 at 17:25
  • 2
    $\begingroup$ @Kjetil Certainly. Not very interesting, are they? ;-) $\endgroup$
    – whuber
    Commented Feb 4, 2023 at 18:00
  • 2
    $\begingroup$ For specific distributions more interesting functions can be possible. For instance, $Y = \sin(2\pi X)$ is independent from $X$ if the domain of $X$ is integer values only. But yes, it will still be the not so interesting map where all values in the domain of $X$ need to be mapped to a single point. It is because $f(X)$ given $X=x$ is a singular distribution in the point $f(x)$. Independence requires that the distribution of $f(X)$ is the same distribution for every value of $X$, $f(x)$ must be a single value for every $x$. $\endgroup$ Commented Feb 4, 2023 at 20:55

5 Answers 5

11
$\begingroup$

Let me write this in a generic manner:

Two random variables for which the expectation of the product is equal to the product of expectations need not be independent.

Let $X\sim \mathcal U(-a, a) $ where $a\in\mathbb R_+$ and let $Y\sim X^2.$ Then as noted $$ \mathbb E[XY]=\mathbb E[X^3] =0=\mathbb E[X]\mathbb E[Y].$$ But they are definitely not independent.


Reference:

$\rm [I]$ Counterexamples in Probability and Statistics, Joseph P. Romano, Andrew F. Siegel, Wadsworth, $1986, $ ex. $4.15.$

$\endgroup$
11
$\begingroup$

Since $Y = X^2$, $[-1/2 \leq X \leq 0] \cap [Y > 1/4] = \varnothing$, hence $P[-1/2 \leq X \leq 0, Y > 1/4] = 0$. On the other hand, $P[-1/2 \leq X \leq 0] = 1/4 > 0, P[Y > 1/4] = 1/2 > 0$. Hence the independence defining relation $P(A \cap B) = P(A)P(B)$ for all $A \in \sigma(X), B \in \sigma(Y)$ fails to hold for $A = [-1/2 \leq X \leq 0]$ and $B = [Y > 1/4]$, i.e., $X$ and $Y$ are not independent.

Above is a formal proof. Intuitively, since $Y$ is a deterministic function of $X$, knowing the value of $X$ means knowing the value of $Y$, hence $Y$ and $X$ of course cannot be independent (the heuristic definition of independence between $X$ and $Y$ requires that observing the information provided by $X$ does not increase the information of $Y$).

It is worth emphasizing that $E[XY] = E[X]E[Y]$ is a necessary condition, rather than a sufficient condition for the independence of $X$ and $Y$. If you really want to express the independence of $X$ and $Y$ in terms of expectations, it should be stated as

$X$ and $Y$ are independent $\iff$ $E[f(X)g(Y)] = E[f(X)]E[g(Y)]$ for all Borel measurable functions $f$ and $g$.

Evidently, the condition $E[f(X)g(Y)] = E[f(X)]E[g(Y)]$ is much stronger than the condition $E[XY] = E[X]E[Y]$. The former is a system of infinitely many equations, while the latter is a single equation.

$\endgroup$
7
$\begingroup$

You need to distinguish between the concepts of uncorrelatedness, which involves the decomposition of the expectation of a product, and independence, which involves the decomposition of a joint probability distribution. What you're calling independence is actually uncorrelatedness. What happens, as the other answers show, is that independence is a stronger condition than uncorrelatedness, that is, independence implies uncorrelatedness but the vice versa is not true.

And your example is exactly one example that is used to show that uncorrelatedness does not imply independence.

$\endgroup$
0
3
$\begingroup$

They are clearly not independent: if you know the value $X$ then this gives you information about the value of $Y$ and indeed allows you to calculate it precisely.

You have correctly said independence means $P(X\le a, Y\le b)=P(X\le a) P(Y\le b)$, so for example consider $a=\frac13$ and $b=\frac 14$.

  • $X \le \frac 13$, i.e. when $-1 \le X \le \frac13$, has probability $\frac23$
  • $Y \le \frac 14$, i.e. when $-\frac12 \le X \le \frac12$, has probability $\frac12$
  • $X \le \frac 13, Y \le \frac 14$ together, i.e. when $-\frac12 \le X \le \frac13$, has probability $\frac5{12}$

but $\frac23 \times \frac12 \not=\frac5{12}$ so they are not independent. Try other examples with $-1 < a < 1$ and $0 < b < 1$.

$E[XY] = E[X]E[Y]$ is a condition for zero covariance and so zero correlation, but this on its own does not imply independence. The bottom row of Wikipedia's chart illustrating correaltion has other examples of this; in your question you would get a parabolic U-shaped chart.

$\endgroup$
1
  • 2
    $\begingroup$ Note that zero covariance is sufficient for independence if $X$ and $Y$ are jointly Gaussian, but not necessarily (or maybe ever) for other distributions. That it works for multivariate Gaussian distributions might be the source of the confusion. $\endgroup$
    – Dave
    Commented Feb 4, 2023 at 14:03
2
$\begingroup$

Here is a simulation of how the joint distribution looks like:

example

It shows clearly that the two variables are not independent.

Your formula is more generally (including dependent variables).

$$\text{E}[XY] = \text{E}[X]\text{E}[Y] + \text{COV}(X,Y)$$

So when you compute $\text{E}[XY] = \text{E}[X]\text{E}[Y]$ then it must mean that $\text{COV}(X,Y) = 0$. But that doesn't imply independence. This makes your case an example of: Why zero correlation does not necessarily imply independence

$\endgroup$
1
  • $\begingroup$ It is not clear where you got the idea for the product $\text{E}[XY] = \text{E}[X]\text{E}[Y]$. Possibly you are confusing it for a description of independence as $P(A \text{ and } B) = P(A) \cdot P(B)$ $\endgroup$ Commented Feb 4, 2023 at 19:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.