9
$\begingroup$

Is it true that if $X_1, X_2, \ldots ,X_n$ are independent random variables, then \begin{align} & f_{X_1,X_2,\ldots,X_n}(x_1,x_2,\ldots,x_n) \\ = {} & f_{X_1}(x_1)\times f_{X_2}(x_2) \times \cdots \times f_{X_n}(x_n) \end{align} (i.e. joint probability density of independent random variables is equal to the product of marginal densities) ?

If so, what is the proof behind this theorem (or should the statement be treated as the definition of independence, rather than a theorem)? It's not a homework, I am asking because I am curious what the proof is.

Thank you,

$\endgroup$
2
  • 1
    $\begingroup$ It cannot be a definition. Some random variable do not have a density. A definition should make sense in general. The definition is given in @Zen's answer. $\endgroup$ Sep 9, 2017 at 10:28
  • $\begingroup$ Is the converse of this statement true? That is, does joint density = product of marginal densities imply independence? $\endgroup$
    – Zslice
    Apr 8, 2018 at 16:01

3 Answers 3

16
$\begingroup$

By definition, the random variables $X_1,\dots,X_n$ are independent iff $$ \Pr(X_1\in B_1,\dots,X_n\in B_n) = \Pr(X_1\in B_1)\dots\Pr(X_n\in B_n) $$ for every choice of Borel sets $B_1,\dots,B_n$. Hence, picking $B_i=(-\infty,t_i]$, we have $$ \Pr(X_1\leq t_1,\dots,X_n\leq t_n) = \Pr(X_1\leq t_1)\dots\Pr(X_n\leq t_n). \qquad (*) $$ If each $X_i$ has a density $f_{X_i}$, then the RHS of $(*)$ is equal to $$ \left(\int_{-\infty}^{t_1} f_{X_1}(x_1)\,dx_1\right) \dots \left(\int_{-\infty}^{t_n} f_{X_n}(x_n)\,dx_n\right). $$ By Fubini's theorem, this is equal to $$ \int_{-\infty}^{t_n}\dots\int_{-\infty}^{t_1} f_{X_1}(x_1)\dots f_{X_n}(x_n)\,dx_1\dots dx_n. $$ Hence, it follows that the random vector $(X_1,\dots,X_n)$ has density $$ f_{X_1,\dots X_n}(x_1,\dots,x_n) = f_{X_1}(x_1)\dots f_{X_n}(x_n). $$

$\endgroup$
2
  • $\begingroup$ You cannot use the equation that you're trying to prove, inside the proof itself. Why is the joint equal to the product of the marginals? Why not the sum of marginals? I think you have yo use conditional probability to prove this. $\endgroup$
    – Cybernetic
    Mar 2, 2020 at 15:21
  • 1
    $\begingroup$ Absurd. There is no petitio principii in the argument above. $\endgroup$
    – Zen
    Mar 2, 2020 at 16:40
3
$\begingroup$

Informally: $$ \Pr(X \in \mathrm{d}x, Y \in \mathrm{d}y) = \Pr(X \in \mathrm{d}x) \Pr(Y \in \mathrm{d}y) = \bigl(f_X(x)\mathrm{d}x\bigr)\bigl(f_Y(y)\mathrm{d}y\bigr) = f_X(x)f_Y(y)\mathrm{d}x\mathrm{d}y. $$

$\endgroup$
1
  • $\begingroup$ This just restates the OP question as an equation. $\endgroup$
    – Cybernetic
    Mar 2, 2020 at 15:20
1
$\begingroup$

By definition:

$f_{X,Y}(x,y) = f_{X\mid Y}(x\mid y)f_Y(y)$

If $X$ and $Y$ are independent:

$f_{X\mid Y}(x\mid y) = f_X(x)$

Therefore

$f_{X,Y}(x,y) = f_X(x)f_Y(y)$

Or more generally for the multinomial case:

$\begin{align} f(x_1, \dots, x_n) & = f(x_1, \dots, x_n) \\ & = f(x_1 \mid x_2, \dots, x_n) p(x_2, \dots, x_n) \\ & = f(x_1 \mid x_2, \dots, x_n) f(x_2 \mid x_3, \dots, x_n) f(x_3, \dots, x_n) \\ & = \dots \\ & = f(x_1 \mid x_2, \dots, x_n) f(x_2 \mid x_3, \dots, x_n) \dots f(x_{n-1} \mid x_n) f(x_n) \\ & = f(x_1)...f(x_n) \text{ (By Independence)}\\ \end{align}$

$\endgroup$
4
  • 3
    $\begingroup$ I think the logic of this argument is not correct. When it says that $f_{X\mid Y}(x\mid y) = f_X(x)$ it's basically using what it's trying to prove. See the answer bellow. $\endgroup$
    – Zen
    Sep 9, 2017 at 2:31
  • $\begingroup$ Really? He's trying to prove that a joint is the product of marginals under independence. What in that line is a joint distribution? $\endgroup$
    – Dale C
    Sep 9, 2017 at 3:08
  • 4
    $\begingroup$ I disagree with "$f_{X,Y}(x,y) = f_{X\mid Y}(x\mid y)f_Y(y)$ by definition". The conditional density is a mathematical object of higher level than the joint density. $\endgroup$ Sep 9, 2017 at 10:23
  • 2
    $\begingroup$ @Tielfish Poele: in this argument, $f_{X\mid Y}(x\mid y)=f_X(x)$ follows from the first formula $f_{X,Y}(x,y)=f_{X\mid Y}(x\mid y)f_Y(y)$ if you use the fact that $f_{X,Y}(x,y)=f_X(x) f_Y(y)$ for independent $X$ and $Y$ (and take care of possible division by zero). But you can't use this fact, because you're trying to prove it. $\endgroup$
    – Zen
    Sep 10, 2017 at 13:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.