# Functions of Independent Random Variables

Is the claim that functions of independent random variables are themselves independent, true?

I have seen that result often used implicitly in some proofs, for example in the proof of independence between the sample mean and the sample variance of a normal distribution, but I have not been able to find justification for it. It seems that some authors take it as given but I am not certain that this is always the case.

## 4 Answers

The most general and abstract definition of independence makes this assertion trivial while supplying an important qualifying condition: that two random variables are independent means the sigma-algebras they generate are independent. Because the sigma-algebra generated by a measurable function of a sigma-algebra is a sub-algebra, a fortiori any measurable functions of those random variables have independent algebras, whence those functions are independent.

(When a function is not measurable, it usually does not create a new random variable, so the concept of independent wouldn't even apply.)

Let's unwrap the definitions to see how simple this is. Recall that a random variable $X$ is a real-valued function defined on the "sample space" $\Omega$ (the set of outcomes being studied via probability).

1. A random variable $X$ is studied by means of the probabilities that its value lies within various intervals of real numbers (or, more generally, sets constructed in simple ways out of intervals: these are the Borel measurable sets of real numbers).

2. Corresponding to any Borel measurable set $I$ is the event $X^{*}(I)$ consisting of all outcomes $\omega$ for which $X(\omega)$ lies in $I$.

3. The sigma-algebra generated by $X$ is determined by the collection of all such events.

4. The naive definition says two random variables $X$ and $Y$ are independent "when their probabilities multiply." That is, when $I$ is one Borel measurable set and $J$ is another, then

$\Pr(X(\omega)\in I\text{ and }Y(\omega)\in J) = \Pr(X(\omega)\in I)\Pr(Y(\omega)\in J).$

5. But in the language of events (and sigma algebras) that's the same as

$\Pr(\omega \in X^{*}(I)\text{ and }\omega \in Y^{*}(J)) = \Pr(\omega\in X^{*}(I))\Pr(\omega\in Y^{*}(J)).$

Consider now two functions $f, g:\mathbb{R}\to\mathbb{R}$ and suppose that $f \circ X$ and $g\circ Y$ are random variables. (The circle is functional composition: $(f\circ X)(\omega) = f(X(\omega))$. This is what it means for $f$ to be a "function of a random variable".) Notice--this is just elementary set theory--that

$$(f\circ X)^{*}(I) = X^{*}(f^{*}(I)).$$

In other words, every event generated by $f\circ X$ (which is on the left) is automatically an event generated by $X$ (as exhibited by the form of the right hand side). Therefore (5) automatically holds for $f\circ X$ and $g\circ Y$: there's nothing to check!

NB You may replace "real-valued" everywhere by "with values in $\mathbb{R}^d$" without needing to change anything else in any material way. This covers the case of vector-valued random variables.

• Sigma algebras are advanced (graduate level) stuff. – Aksakal Apr 23 '14 at 14:54
• @Aksakal It depends on what school you go to or what books you read. (I have successfully taught this material at the second-year undergraduate level. There also are wonderfully accessible accounts of this theory at the undergraduate level, such as Steven Shreve's texts on stochastic calculus, which are addressed to students with just a calculus background.) But how is that relevant? Any justification--even a sophisticated one--should be preferred to an unjustified assertion. – whuber Apr 23 '14 at 14:59
• You are very kind to go to all that trouble to help someone who asked a question. Thanks again. And you are right, the definitions are not too daunting after all. – JohnK Apr 23 '14 at 15:23
• Which undergraduate book would you recommend on this topic? I see multiple books by Shreve. – Joe Oct 30 '20 at 3:55
• @Joe I like Shreve's books a lot for their clarity and insight. The first one, on the Binomial Asset Pricing Model, gets the key ideas across without using any Calculus or measure theory. The second one recapitulates that approach but using Calculus and introducing measure theory. – whuber Oct 30 '20 at 13:05

Consider this "less advanced" proof:

Let $X:\Omega_X\to\mathbb{R}^n,Y:\Omega_Y\to\mathbb{R}^m,f:\mathbb{R}^n\to\mathbb{R}^k,g:\mathbb{R}^m\to\mathbb{R}^p$, where $X,Y$ are independent random variables and $f,g$ are measurable functions. Then: $$P\{f(X)\leq x \text{ and } g(Y)\leq y\}\\=P(\{f(X)\leq x\}\cap\{g(Y)\leq y\})\\=P(\{X\in\{w\in\mathbb{R}^n:f(w)\leq x\}\}\cap\{Y\in\{w\in\mathbb{R}^m:g(w)\leq y\}\}).$$ Using independence of $X$ and $Y$, $$P(\{X\in\{w\in\mathbb{R}^n:f(w)\leq x\}\}\cap\{Y\in\{w\in\mathbb{R}^m:g(w)\leq y\}\})=\\=P\{X\in\{w\in\mathbb{R}^n:f(w)\leq x\}\cdot P\{Y\in\{w\in\mathbb{R}^m:g(w)\leq y\}\} \\=P\{f(X)\leq x\}\cdot P\{g(Y)\leq y\}.$$

The idea is to notice that the set $$\{f(X)\leq x\}\equiv\{w\in\Omega_X:f(X(w))\leq x\}=\{X\in\{w\in\mathbb{R}^n:f(w)\leq x\}\},$$ so properties that are valid for $X$ are extended to $f(X)$ and the same happens for $Y$.

• +1. Thank you for this contribution, which so clearly focuses on the essential idea. Welcome to our site! – whuber Sep 20 '15 at 13:26

Yes, $g(X)$ and $h(Y)$ are independent for any functions $g$ and $h$ so long as $X$ and $Y$ are independent. It's a very well known results, which is studied in probability theory courses. I'm sure you can find it in any standard text like Billingsley's.

• Thanks, I am currently studying Hogg & Craig and MGB. Billingsley is the next logical step. – JohnK Apr 23 '14 at 15:02
• Billinglsey's a torture unless you're mathematician and already studied measures. Partarathy's intro is much easier 2-in-1 book, Alan Karr's Probability text is also easy read. – Aksakal Apr 23 '14 at 15:28
• Another easier text than Billingsley's: probability.ca/jeff/grprobbook.html – Adrian Jun 15 '19 at 1:50

Not as an alternative, but as an addition to the previous brilliant answers, note that this result is in fact very intuitive.

Usually, we think that $$X$$ and $$Y$$ being independent means that knowing the value of $$X$$ gives no information about the value of $$Y$$ and vice versa. This interpretation obviously implies that you can't somehow "squeeze" an information out by applying a function (or by any other means actually).