4
$\begingroup$

Is it true that if $X_1, X_2, \ldots ,X_n$ are independent random variables, then \begin{align} & f_{X_1,X_2,\ldots,X_n}(x_1,x_2,\ldots,x_n) \\ = {} & f_{X_1}(x_1)\times f_{X_2}(x_2) \times \cdots \times f_{X_n}(x_n) \end{align} (i.e. joint probability density of independent random variables is equal to the product of marginal densities) ?

If so, what is the proof behind this theorem (or should the statement be treated as the definition of independence, rather than a theorem)? It's not a homework, I am asking because I am curious what the proof is.

Thank you,

$\endgroup$
  • 1
    $\begingroup$ It cannot be a definition. Some random variable do not have a density. A definition should make sense in general. The definition is given in @Zen's answer. $\endgroup$ – Stéphane Laurent Sep 9 '17 at 10:28
  • $\begingroup$ Is the converse of this statement true? That is, does joint density = product of marginal densities imply independence? $\endgroup$ – Zslice Apr 8 '18 at 16:01
13
$\begingroup$

By definition, the random variables $X_1,\dots,X_n$ are independent iff $$ \Pr(X_1\in B_1,\dots,X_n\in B_n) = \Pr(X_1\in B_1)\dots\Pr(X_n\in B_n) $$ for every choice of Borel sets $B_1,\dots,B_n$. Hence, picking $B_i=(-\infty,t_i]$, we have $$ \Pr(X_1\leq t_1,\dots,X_n\leq t_n) = \Pr(X_1\leq t_1)\dots\Pr(X_n\leq t_n). \qquad (*) $$ If each $X_i$ has a density $f_{X_i}$, then the RHS of $(*)$ is equal to $$ \left(\int_{-\infty}^{t_1} f_{X_1}(x_1)\,dx_1\right) \dots \left(\int_{-\infty}^{t_n} f_{X_n}(x_n)\,dx_n\right). $$ By Fubini's theorem, this is equal to $$ \int_{-\infty}^{t_n}\dots\int_{-\infty}^{t_1} f_{X_1}(x_1)\dots f_{X_n}(x_n)\,dx_1\dots dx_n. $$ Hence, it follows that the random vector $(X_1,\dots,X_n)$ has density $$ f_{X_1,\dots X_n}(x_1,\dots,x_n) = f_{X_1}(x_1)\dots f_{X_n}(x_n). $$

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ You cannot use the equation that you're trying to prove, inside the proof itself. Why is the joint equal to the product of the marginals? Why not the sum of marginals? I think you have yo use conditional probability to prove this. $\endgroup$ – Cybernetic Mar 2 at 15:21
  • $\begingroup$ Absurd. There is no petitio principii in the argument above. $\endgroup$ – Zen Mar 2 at 16:40
3
$\begingroup$

Informally: $$ \Pr(X \in \mathrm{d}x, Y \in \mathrm{d}y) = \Pr(X \in \mathrm{d}x) \Pr(Y \in \mathrm{d}y) = \bigl(f_X(x)\mathrm{d}x\bigr)\bigl(f_Y(y)\mathrm{d}y\bigr) = f_X(x)f_Y(y)\mathrm{d}x\mathrm{d}y. $$

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ This just restates the OP question as an equation. $\endgroup$ – Cybernetic Mar 2 at 15:20
1
$\begingroup$

By definition:

$f_{X,Y}(x,y) = f_{X\mid Y}(x\mid y)f_Y(y)$

If $X$ and $Y$ are independent:

$f_{X\mid Y}(x\mid y) = f_X(x)$

Therefore

$f_{X,Y}(x,y) = f_X(x)f_Y(y)$

Or more generally for the multinomial case:

$\begin{align} f(x_1, \dots, x_n) & = f(x_1, \dots, x_n) \\ & = f(x_1 \mid x_2, \dots, x_n) p(x_2, \dots, x_n) \\ & = f(x_1 \mid x_2, \dots, x_n) f(x_2 \mid x_3, \dots, x_n) f(x_3, \dots, x_n) \\ & = \dots \\ & = f(x_1 \mid x_2, \dots, x_n) f(x_2 \mid x_3, \dots, x_n) \dots f(x_{n-1} \mid x_n) f(x_n) \\ & = f(x_1)...f(x_n) \text{ (By Independence)}\\ \end{align}$

| cite | improve this answer | |
$\endgroup$
  • 3
    $\begingroup$ I think the logic of this argument is not correct. When it says that $f_{X\mid Y}(x\mid y) = f_X(x)$ it's basically using what it's trying to prove. See the answer bellow. $\endgroup$ – Zen Sep 9 '17 at 2:31
  • $\begingroup$ Really? He's trying to prove that a joint is the product of marginals under independence. What in that line is a joint distribution? $\endgroup$ – Dale C Sep 9 '17 at 3:08
  • 4
    $\begingroup$ I disagree with "$f_{X,Y}(x,y) = f_{X\mid Y}(x\mid y)f_Y(y)$ by definition". The conditional density is a mathematical object of higher level than the joint density. $\endgroup$ – Stéphane Laurent Sep 9 '17 at 10:23
  • 2
    $\begingroup$ @Tielfish Poele: in this argument, $f_{X\mid Y}(x\mid y)=f_X(x)$ follows from the first formula $f_{X,Y}(x,y)=f_{X\mid Y}(x\mid y)f_Y(y)$ if you use the fact that $f_{X,Y}(x,y)=f_X(x) f_Y(y)$ for independent $X$ and $Y$ (and take care of possible division by zero). But you can't use this fact, because you're trying to prove it. $\endgroup$ – Zen Sep 10 '17 at 13:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.