4
$\begingroup$

Is it true that if $X_1, X_2, \ldots ,X_n$ are independent random variables, then \begin{align} & f_{X_1,X_2,\ldots,X_n}(x_1,x_2,\ldots,x_n) \\ = {} & f_{X_1}(x_1)\times f_{X_2}(x_2) \times \cdots \times f_{X_n}(x_n) \end{align} (i.e. joint probability density of independent random variables is equal to the product of marginal densities) ?

If so, what is the proof behind this theorem (or should the statement be treated as the definition of independence, rather than a theorem)? It's not a homework, I am asking because I am curious what the proof is.

Thank you,

$\endgroup$
  • 1
    $\begingroup$ It cannot be a definition. Some random variable do not have a density. A definition should make sense in general. The definition is given in @Zen's answer. $\endgroup$ – Stéphane Laurent Sep 9 '17 at 10:28
  • $\begingroup$ Is the converse of this statement true? That is, does joint density = product of marginal densities imply independence? $\endgroup$ – Zslice Apr 8 '18 at 16:01
13
$\begingroup$

By definition, the random variables $X_1,\dots,X_n$ are independent iff $$ \Pr(X_1\in B_1,\dots,X_n\in B_n) = \Pr(X_1\in B_1)\dots\Pr(X_n\in B_n) $$ for every choice of Borel sets $B_1,\dots,B_n$. Hence, picking $B_i=(-\infty,t_i]$, we have $$ \Pr(X_1\leq t_1,\dots,X_n\leq t_n) = \Pr(X_1\leq t_1)\dots\Pr(X_n\leq t_n). \qquad (*) $$ If each $X_i$ has a density $f_{X_i}$, then the RHS of $(*)$ is equal to $$ \left(\int_{-\infty}^{t_1} f_{X_1}(x_1)\,dx_1\right) \dots \left(\int_{-\infty}^{t_n} f_{X_n}(x_n)\,dx_n\right). $$ By Fubini's theorem, this is equal to $$ \int_{-\infty}^{t_n}\dots\int_{-\infty}^{t_1} f_{X_1}(x_1)\dots f_{X_n}(x_n)\,dx_1\dots dx_n. $$ Hence, it follows that the random vector $(X_1,\dots,X_n)$ has density $$ f_{X_1,\dots X_n}(x_1,\dots,x_n) = f_{X_1}(x_1)\dots f_{X_n}(x_n). $$

$\endgroup$
3
$\begingroup$

Informally: $$ \Pr(X \in \mathrm{d}x, Y \in \mathrm{d}y) = \Pr(X \in \mathrm{d}x) \Pr(Y \in \mathrm{d}y) = \bigl(f_X(x)\mathrm{d}x\bigr)\bigl(f_Y(y)\mathrm{d}y\bigr) = f_X(x)f_Y(y)\mathrm{d}x\mathrm{d}y. $$

$\endgroup$
1
$\begingroup$

By definition:

$f_{X,Y}(x,y) = f_{X\mid Y}(x\mid y)f_Y(y)$

If $X$ and $Y$ are independent:

$f_{X\mid Y}(x\mid y) = f_X(x)$

Therefore

$f_{X,Y}(x,y) = f_X(x)f_Y(y)$

Or more generally for the multinomial case:

$\begin{align} f(x_1, \dots, x_n) & = f(x_1, \dots, x_n) \\ & = f(x_1 \mid x_2, \dots, x_n) p(x_2, \dots, x_n) \\ & = f(x_1 \mid x_2, \dots, x_n) f(x_2 \mid x_3, \dots, x_n) f(x_3, \dots, x_n) \\ & = \dots \\ & = f(x_1 \mid x_2, \dots, x_n) f(x_2 \mid x_3, \dots, x_n) \dots f(x_{n-1} \mid x_n) f(x_n) \\ & = f(x_1)...f(x_n) \text{ (By Independence)}\\ \end{align}$

$\endgroup$
  • 3
    $\begingroup$ I think the logic of this argument is not correct. When it says that $f_{X\mid Y}(x\mid y) = f_X(x)$ it's basically using what it's trying to prove. See the answer bellow. $\endgroup$ – Zen Sep 9 '17 at 2:31
  • $\begingroup$ Really? He's trying to prove that a joint is the product of marginals under independence. What in that line is a joint distribution? $\endgroup$ – Tilefish Poele Sep 9 '17 at 3:08
  • 4
    $\begingroup$ I disagree with "$f_{X,Y}(x,y) = f_{X\mid Y}(x\mid y)f_Y(y)$ by definition". The conditional density is a mathematical object of higher level than the joint density. $\endgroup$ – Stéphane Laurent Sep 9 '17 at 10:23
  • 2
    $\begingroup$ @Tielfish Poele: in this argument, $f_{X\mid Y}(x\mid y)=f_X(x)$ follows from the first formula $f_{X,Y}(x,y)=f_{X\mid Y}(x\mid y)f_Y(y)$ if you use the fact that $f_{X,Y}(x,y)=f_X(x) f_Y(y)$ for independent $X$ and $Y$ (and take care of possible division by zero). But you can't use this fact, because you're trying to prove it. $\endgroup$ – Zen Sep 10 '17 at 13:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.