# Are linear combinations of independent random variables again independent?

Let $$X_1,X_2,\ldots,X_n$$ be (iid) Random variables and define $$Y_n:=\sum_{j=1}^na_jX_j$$ with $$a_j\in \mathbb{R}$$, can we then say that the $$a_jX_j$$ are independent aswell. Can we express the MGF than in the following way $$M_{Y_n}(t)=\mathbb{E}(e^{t(a_1X_1+\ldots a_nX_n)})=\mathbb{E}(e^{ta_1X_1}\cdot\ldots\cdot e^{t a_nX_n})=M_{X_1}(a_1t)\cdot\ldots \cdot M_{X_n}(a_nt)$$

New contributor
muhammed gunes is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.
• Independent of what? If $X_1,X_2,\ldots,X_n$ are $n$ (iid) random variables (or even just plain independent (but not necessarily identically distributed) random variables), then so are $a_1X_1, a_2X_2,\ldots, a_nX_n$ a collection of $n$ independent but not necessarily identically. distributed random variables. May 21 at 18:27
• @DilipSarwate They are identically independent distributed Random variables. So why are all $a_jX_j$ independent? May 21 at 18:33
• Independence of $X_1, X_2,\ldots,X_n$ means that the CDF $F_{X_1, X_2,\ldots,X_n}(x_1, x_2,\ldots,x_n)$ factors into $F_{X_1}(x_1)F_{X_2}(x_2)\cdots F_{X_n}(x_n)$. Since $$F_{a_iX_i}(\alpha_i)=P(a_iX_i\leq\alpha_i)=P\left(X_i\leq\frac{\alpha_i}{a_i}\right)=F_{X_i}\left(\frac{\alpha_i}{a_i}\right)$$, can you prove that the factorization of the joint CDF of the $X_i$ into the product of the marginal CDFs of the $X_i$implies that the joint CDF of the $a_iX_i$ factors into the product of the marginal CDFs of the $a_iX_i$ which in turn implies that the $a_iX_i$ are independent random variables? May 21 at 19:02
• tl;dr, yes, they're independent and you can factorize the mgf. any deterministic and univariate function of mutually independent set RVs are again mutually independent. See this thread for more information: stats.stackexchange.com/questions/94872/… May 21 at 19:47
• The title is a bit off from what you really intended to ask. The "linear combination" is just a single r.v. and it is pointless to speak the independence of a single r.v. It's better to reframe your title as "Are linear (or more specifically, scale) transformation of independent r.v.s again independent?" May 22 at 1:46

Yes, for the content of your question. and No, for the title, in general.

Yes:

Your $$a_1,\dots, a_n$$ are just some constant numbers. Then the independence of $$X_1,\dots, X_n$$ implies that the $$a_iX_i$$ are also independent. In fact, for any functions $$g_1,\dots, g_n$$ you would find that $$g_1(X_1)$$, $$g_2(X_2)\dots g_n(X_n)$$ are independent.

From independence it follows that $$E\prod g_i(X_i) = \prod E g_i(X_i)$$. Your calculations are correct.

No:

Now look at $$Y = \sum_{i=1}^n a_iX_i$$ and $$Z = \sum_{i=1}^n b_iX_i$$. The linear combinations $$Y$$ and $$Z$$ of the $$X_i$$ are in general not independent. They are independent when $$b_i=0$$ for all $$i$$ where $$a_i\neq 0$$, and $$a_j=0$$ for all $$j$$ with $$b_j\neq 0$$. The simplest case where this is not fulfilled is $$a_i=b_i$$ for all $$i$$: then $$Y=Z$$.

###### Normal distributed variables $$X_i$$

When the $$X_i$$ are normal distributed, the condition "$$a_i\neq 0 \implies b_i=0$$" can be relaxed. In that case the linear combinations $$Y$$ and $$Z$$ are independent whenever $$\sum a_ib_i = 0$$, i.e., when the vectors $$a=(a_1,\dots, a_n)$$ and $$b=(b_1,\dots,b_n)$$ are orthogonal. This is a fact that contributes to the popularity of the normal distribution for modelling. One of the consequences of this fact is, that estimates $$\bar x$$ for the mean and $$s^2$$ for the variance of normal distributed r.v. are independent :-)

New contributor
Ute is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.
• I upvoted especially for the first sentence, which directly addresses the OP's confusion. Minor quibble: the "if and only if" in "They are independent if and only if $b_i=0$ for all $i$ where $a_i\neq 0$, and $a_j=0$ for all $j$ with $b_j\neq 0$." is not entirely true, although close enough. And the normal distribution is not the only exception, although it's the most commonly encountered.
– Stef
May 23 at 8:29
• yes, @Stef, you are competely right, I oversimplified too much. Thank you very much for your comment, I already had a kind of bad conscience with my answer! I have sharpened the statements :-)
– Ute
May 23 at 10:55

If $$X_1,...,X_n$$ are mutually independent then $$a_1 X_1,...,a_n X_n$$ are also mutually independent (i.e., using scalar multiples does not get rid of independence). However, the quantity $$Y_n = \sum a_i X_i$$ is typically going to be dependent with $$X_1,...,X_n$$. This means that you can indeed express the MGF pf $$Y_n$$ the way you write it in your post, but if you were to look at the relationship between $$Y_n$$ and any of the $$X_i$$ values, you would have to deal with the dependence between these quantities.

I appreciate the answers here. If we assume that the random variables $$X_1,X_2,\ldots X_n$$ are independent, then the CDF $$F_{X_1,X_2,\ldots X_n}(x_1,x_2\ldots x_n)$$ factors into $$F_{X_1}(x_1)F_{X_2}(x_2)\ldots F_{X_n}(x_n)$$. The CDF of $$Y_n=\sum_{j=1}^{n}a_jX_j$$ is nothing but $$F_{Y_n}(y_n)=F_{a_1X_1,a_2X_2,\ldots a_nX_n}(a_1x_1,a_2x_2,\ldots a_nx_n)=\mathbb{P}(a_1X_1\leq c_1,a_2X_2\leq c_2,\ldots a_nX_n\leq c_n)=\mathbb{P}(X_1\leq \frac{c_1}{a_1},X_2\leq \frac{c_2}{a_2},\ldots X_n\leq \frac{c_n}{a_n})=F_{X_1}(\frac{c_1}{a_1})F_{X_2}(\frac{c_2}{a_2})\ldots F_{X_n}(\frac{c_n}{a_n})$$

by our assumption . So lineair combinations of random variables are random aswell.

New contributor
muhammed gunes is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.
• Although this answer's idea is a good one, in its details it is incorrect because it assumes all the $a_i$ are positive. When any $a_i\lt 0,$ division by $a_i$ reverses the inequality; and when $a_i=0,$ you can't divide by it at all. The result you seek follows immediately from the far more general (yet simple) fact that functions of independent variables remain independent.
– whuber
May 22 at 12:33
• Your calculations are mostly correct, but your concluding sentence "So lineair combinations of random variables are random aswell." is not correct. In order to make it correct your need to be very careful with the vocabulary you use.
– Stef
May 23 at 8:34