Proof that joint probability density of independent random variables is equal to the product of marginal densities

Question

Is it true that if $X_1, X_2, \ldots ,X_n$ are independent random variables, then \begin{align} & f_{X_1,X_2,\ldots,X_n}(x_1,x_2,\ldots,x_n) \\ = {} & f_{X_1}(x_1)\times f_{X_2}(x_2) \times \cdots \times f_{X_n}(x_n) \end{align} (i.e. joint probability density of independent random variables is equal to the product of marginal densities) ?

If so, what is the proof behind this theorem (or should the statement be treated as the definition of independence, rather than a theorem)? It's not a homework, I am asking because I am curious what the proof is.

Thank you,

Answer 1

By definition, the random variables $X_1,\dots,X_n$ are independent iff $$ \Pr(X_1\in B_1,\dots,X_n\in B_n) = \Pr(X_1\in B_1)\dots\Pr(X_n\in B_n) $$ for every choice of Borel sets $B_1,\dots,B_n$. Hence, picking $B_i=(-\infty,t_i]$, we have $$ \Pr(X_1\leq t_1,\dots,X_n\leq t_n) = \Pr(X_1\leq t_1)\dots\Pr(X_n\leq t_n). \qquad (*) $$ If each $X_i$ has a density $f_{X_i}$, then the RHS of $(*)$ is equal to $$ \left(\int_{-\infty}^{t_1} f_{X_1}(x_1)\,dx_1\right) \dots \left(\int_{-\infty}^{t_n} f_{X_n}(x_n)\,dx_n\right). $$ By Fubini's theorem, this is equal to $$ \int_{-\infty}^{t_n}\dots\int_{-\infty}^{t_1} f_{X_1}(x_1)\dots f_{X_n}(x_n)\,dx_1\dots dx_n. $$ Hence, it follows that the random vector $(X_1,\dots,X_n)$ has density $$ f_{X_1,\dots X_n}(x_1,\dots,x_n) = f_{X_1}(x_1)\dots f_{X_n}(x_n). $$

Answer 2

Informally: $$ \Pr(X \in \mathrm{d}x, Y \in \mathrm{d}y) = \Pr(X \in \mathrm{d}x) \Pr(Y \in \mathrm{d}y) = \bigl(f_X(x)\mathrm{d}x\bigr)\bigl(f_Y(y)\mathrm{d}y\bigr) = f_X(x)f_Y(y)\mathrm{d}x\mathrm{d}y. $$

Answer 3

By definition:

$f_{X,Y}(x,y) = f_{X\mid Y}(x\mid y)f_Y(y)$

If $X$ and $Y$ are independent:

$f_{X\mid Y}(x\mid y) = f_X(x)$

Therefore

$f_{X,Y}(x,y) = f_X(x)f_Y(y)$

Or more generally for the multinomial case:

$\begin{align} f(x_1, \dots, x_n) & = f(x_1, \dots, x_n) \\ & = f(x_1 \mid x_2, \dots, x_n) p(x_2, \dots, x_n) \\ & = f(x_1 \mid x_2, \dots, x_n) f(x_2 \mid x_3, \dots, x_n) f(x_3, \dots, x_n) \\ & = \dots \\ & = f(x_1 \mid x_2, \dots, x_n) f(x_2 \mid x_3, \dots, x_n) \dots f(x_{n-1} \mid x_n) f(x_n) \\ & = f(x_1)...f(x_n) \text{ (By Independence)}\\ \end{align}$