11
$\begingroup$

Let $X_1,X_2,\ldots,X_n$ be (iid) Random variables and define $Y_n:=\sum_{j=1}^na_jX_j$ with $a_j\in \mathbb{R}$, can we then say that the $a_jX_j$ are independent aswell. Can we express the MGF than in the following way $$M_{Y_n}(t)=\mathbb{E}(e^{t(a_1X_1+\ldots a_nX_n)})=\mathbb{E}(e^{ta_1X_1}\cdot\ldots\cdot e^{t a_nX_n})=M_{X_1}(a_1t)\cdot\ldots \cdot M_{X_n}(a_nt)$$

$\endgroup$
5
  • 2
    $\begingroup$ Independent of what? If $X_1,X_2,\ldots,X_n$ are $n$ (iid) random variables (or even just plain independent (but not necessarily identically distributed) random variables), then so are $a_1X_1, a_2X_2,\ldots, a_nX_n$ a collection of $n$ independent but not necessarily identically. distributed random variables. $\endgroup$ May 21, 2023 at 18:27
  • $\begingroup$ @DilipSarwate They are identically independent distributed Random variables. So why are all $a_jX_j$ independent? $\endgroup$ May 21, 2023 at 18:33
  • 2
    $\begingroup$ Independence of $X_1, X_2,\ldots,X_n$ means that the CDF $F_{X_1, X_2,\ldots,X_n}(x_1, x_2,\ldots,x_n)$ factors into $F_{X_1}(x_1)F_{X_2}(x_2)\cdots F_{X_n}(x_n)$. Since $$F_{a_iX_i}(\alpha_i)=P(a_iX_i\leq\alpha_i)=P\left(X_i\leq\frac{\alpha_i}{a_i}\right)=F_{X_i}\left(\frac{\alpha_i}{a_i}\right)$$, can you prove that the factorization of the joint CDF of the $X_i$ into the product of the marginal CDFs of the $X_i$implies that the joint CDF of the $a_iX_i$ factors into the product of the marginal CDFs of the $a_iX_i$ which in turn implies that the $a_iX_i$ are independent random variables? $\endgroup$ May 21, 2023 at 19:02
  • 1
    $\begingroup$ tl;dr, yes, they're independent and you can factorize the mgf. any deterministic and univariate function of mutually independent set RVs are again mutually independent. See this thread for more information: stats.stackexchange.com/questions/94872/… $\endgroup$
    – gunes
    May 21, 2023 at 19:47
  • 5
    $\begingroup$ The title is a bit off from what you really intended to ask. The "linear combination" is just a single r.v. and it is pointless to speak the independence of a single r.v. It's better to reframe your title as "Are linear (or more specifically, scale) transformation of independent r.v.s again independent?" $\endgroup$
    – Zhanxiong
    May 22, 2023 at 1:46

3 Answers 3

20
$\begingroup$

Yes, for the content of your question. and No, for the title, in general.

Yes:

Your $a_1,\dots, a_n$ are just some constant numbers. Then the independence of $X_1,\dots, X_n$ implies that the $a_iX_i$ are also independent. In fact, for any functions $g_1,\dots, g_n$ you would find that $g_1(X_1)$, $g_2(X_2)\dots g_n(X_n)$ are independent.

From independence it follows that $E\prod g_i(X_i) = \prod E g_i(X_i)$. Your calculations are correct.

No:

Now look at $Y = \sum_{i=1}^n a_iX_i$ and $Z = \sum_{i=1}^n b_iX_i$. The linear combinations $Y$ and $Z$ of the $X_i$ are in general not independent. They are independent when $b_i=0$ for all $i$ where $a_i\neq 0$, and $a_j=0$ for all $j$ with $b_j\neq 0$. The simplest case where this is not fulfilled is $a_i=b_i$ for all $i$: then $Y=Z$.

Normal distributed variables $X_i$

When the $X_i$ are normal distributed, the condition "$a_i\neq 0 \implies b_i=0$" can be relaxed. In that case the linear combinations $Y$ and $Z$ are independent whenever $\sum a_ib_i = 0$, i.e., when the vectors $a=(a_1,\dots, a_n)$ and $b=(b_1,\dots,b_n)$ are orthogonal. This is a fact that contributes to the popularity of the normal distribution for modelling. One of the consequences of this fact is, that estimates $\bar x$ for the mean and $s^2$ for the variance of normal distributed r.v. are independent :-)

$\endgroup$
2
  • 2
    $\begingroup$ I upvoted especially for the first sentence, which directly addresses the OP's confusion. Minor quibble: the "if and only if" in "They are independent if and only if $b_i=0$ for all $i$ where $a_i\neq 0$, and $a_j=0$ for all $j$ with $b_j\neq 0$." is not entirely true, although close enough. And the normal distribution is not the only exception, although it's the most commonly encountered. $\endgroup$
    – Stef
    May 23, 2023 at 8:29
  • 1
    $\begingroup$ yes, @Stef, you are competely right, I oversimplified too much. Thank you very much for your comment, I already had a kind of bad conscience with my answer! I have sharpened the statements :-) $\endgroup$
    – Ute
    May 23, 2023 at 10:55
8
$\begingroup$

If $X_1,...,X_n$ are mutually independent then $a_1 X_1,...,a_n X_n$ are also mutually independent (i.e., using scalar multiples does not get rid of independence). However, the quantity $Y_n = \sum a_i X_i$ is typically going to be dependent with $X_1,...,X_n$. This means that you can indeed express the MGF of $Y_n$ the way you write it in your post, but if you were to look at the relationship between $Y_n$ and any of the $X_i$ values, you would have to deal with the dependence between these quantities.

$\endgroup$
1
$\begingroup$

I appreciate the answers here. If we assume that the random variables $X_1,X_2,\ldots X_n$ are independent, then the CDF $F_{X_1,X_2,\ldots X_n}(x_1,x_2\ldots x_n)$ factors into $F_{X_1}(x_1)F_{X_2}(x_2)\ldots F_{X_n}(x_n)$. The CDF of $Y_n=\sum_{j=1}^{n}a_jX_j$ is nothing but $F_{Y_n}(y_n)=F_{a_1X_1,a_2X_2,\ldots a_nX_n}(a_1x_1,a_2x_2,\ldots a_nx_n)=\mathbb{P}(a_1X_1\leq c_1,a_2X_2\leq c_2,\ldots a_nX_n\leq c_n)=\mathbb{P}(X_1\leq \frac{c_1}{a_1},X_2\leq \frac{c_2}{a_2},\ldots X_n\leq \frac{c_n}{a_n})=F_{X_1}(\frac{c_1}{a_1})F_{X_2}(\frac{c_2}{a_2})\ldots F_{X_n}(\frac{c_n}{a_n})$

by our assumption . So lineair combinations of random variables are random aswell.

$\endgroup$
2
  • 3
    $\begingroup$ Although this answer's idea is a good one, in its details it is incorrect because it assumes all the $a_i$ are positive. When any $a_i\lt 0,$ division by $a_i$ reverses the inequality; and when $a_i=0,$ you can't divide by it at all. The result you seek follows immediately from the far more general (yet simple) fact that functions of independent variables remain independent. $\endgroup$
    – whuber
    May 22, 2023 at 12:33
  • 3
    $\begingroup$ Your calculations are mostly correct, but your concluding sentence "So lineair combinations of random variables are random aswell." is not correct. In order to make it correct your need to be very careful with the vocabulary you use. $\endgroup$
    – Stef
    May 23, 2023 at 8:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.