47
$\begingroup$

Is the claim that functions of independent random variables are themselves independent, true?

I have seen that result often used implicitly in some proofs, for example in the proof of independence between the sample mean and the sample variance of a normal distribution, but I have not been able to find justification for it. It seems that some authors take it as given but I am not certain that this is always the case.

$\endgroup$
2

4 Answers 4

56
$\begingroup$

The most general and abstract definition of independence makes this assertion trivial while supplying an important qualifying condition: that two random variables are independent means the sigma-algebras they generate are independent. Because the sigma-algebra generated by a measurable function of a sigma-algebra is a sub-algebra, a fortiori any measurable functions of those random variables have independent algebras, whence those functions are independent.

(When a function is not measurable, it usually does not create a new random variable, so the concept of independent wouldn't even apply.)


Let's unwrap the definitions to see how simple this is. Recall that a random variable $X$ is a real-valued function defined on the "sample space" $\Omega$ (the set of outcomes being studied via probability).

  1. A random variable $X$ is studied by means of the probabilities that its value lies within various intervals of real numbers (or, more generally, sets constructed in simple ways out of intervals: these are the Borel measurable sets of real numbers).

  2. Corresponding to any Borel measurable set $I$ is the event $X^{*}(I)$ consisting of all outcomes $\omega$ for which $X(\omega)$ lies in $I$.

  3. The sigma-algebra generated by $X$ is determined by the collection of all such events.

  4. The naive definition says two random variables $X$ and $Y$ are independent "when their probabilities multiply." That is, when $I$ is one Borel measurable set and $J$ is another, then

    $\Pr(X(\omega)\in I\text{ and }Y(\omega)\in J) = \Pr(X(\omega)\in I)\Pr(Y(\omega)\in J).$

  5. But in the language of events (and sigma algebras) that's the same as

    $\Pr(\omega \in X^{*}(I)\text{ and }\omega \in Y^{*}(J)) = \Pr(\omega\in X^{*}(I))\Pr(\omega\in Y^{*}(J)).$

Consider now two functions $f, g:\mathbb{R}\to\mathbb{R}$ and suppose that $f \circ X$ and $g\circ Y$ are random variables. (The circle is functional composition: $(f\circ X)(\omega) = f(X(\omega))$. This is what it means for $f$ to be a "function of a random variable".) Notice--this is just elementary set theory--that

$$(f\circ X)^{*}(I) = X^{*}(f^{*}(I)).$$

In other words, every event generated by $f\circ X$ (which is on the left) is automatically an event generated by $X$ (as exhibited by the form of the right hand side). Therefore (5) automatically holds for $f\circ X$ and $g\circ Y$: there's nothing to check!


NB You may replace "real-valued" everywhere by "with values in $\mathbb{R}^d$" without needing to change anything else in any material way. This covers the case of vector-valued random variables.

$\endgroup$
6
  • 4
    $\begingroup$ Sigma algebras are advanced (graduate level) stuff. $\endgroup$
    – Aksakal
    Commented Apr 23, 2014 at 14:54
  • 11
    $\begingroup$ @Aksakal It depends on what school you go to or what books you read. (I have successfully taught this material at the second-year undergraduate level. There also are wonderfully accessible accounts of this theory at the undergraduate level, such as Steven Shreve's texts on stochastic calculus, which are addressed to students with just a calculus background.) But how is that relevant? Any justification--even a sophisticated one--should be preferred to an unjustified assertion. $\endgroup$
    – whuber
    Commented Apr 23, 2014 at 14:59
  • 2
    $\begingroup$ You are very kind to go to all that trouble to help someone who asked a question. Thanks again. And you are right, the definitions are not too daunting after all. $\endgroup$
    – JohnK
    Commented Apr 23, 2014 at 15:23
  • 3
    $\begingroup$ @Joe I like Shreve's books a lot for their clarity and insight. The first one, on the Binomial Asset Pricing Model, gets the key ideas across without using any Calculus or measure theory. The second one recapitulates that approach but using Calculus and introducing measure theory. $\endgroup$
    – whuber
    Commented Oct 30, 2020 at 13:05
  • 1
    $\begingroup$ I have also been taught sigma algebras at the second-year undergraduate level. (How successful that was I am not sure; I have forgotten all of it by now.) The study programme was econometrics. $\endgroup$ Commented Jan 7, 2021 at 8:19
26
$\begingroup$

Consider this "less advanced" proof:

Let $X:\Omega_X\to\mathbb{R}^n,Y:\Omega_Y\to\mathbb{R}^m,f:\mathbb{R}^n\to\mathbb{R}^k,g:\mathbb{R}^m\to\mathbb{R}^p$, where $X,Y$ are independent random variables and $f,g$ are measurable functions. Then: $$ P\{f(X)\leq x \text{ and } g(Y)\leq y\}\\ = P(\{f(X)\leq x\}\cap\{g(Y)\leq y\})\\ = P(\{X\in\{w\in\mathbb{R}^n:f(w)\leq x\}\}\cap\{Y\in\{w\in\mathbb{R}^m:g(w)\leq y\}\}). $$ Using independence of $X$ and $Y$, $$ P(\{X\in\{w\in\mathbb{R}^n:f(w)\leq x\}\}\cap\{Y\in\{w\in\mathbb{R}^m:g(w)\leq y\}\})\\ = P\{X\in\{w\in\mathbb{R}^n:f(w)\leq x\}\}\cdot P\{Y\in\{w\in\mathbb{R}^m:g(w)\leq y\}\}\\ = P\{f(X)\leq x\}\cdot P\{g(Y)\leq y\}. $$

The idea is to notice that the set $$ \{f(X)\leq x\}\equiv\{w\in\Omega_X:f(X(w))\leq x\}=\{X\in\{w\in\mathbb{R}^n:f(w)\leq x\}\}, $$ so properties that are valid for $X$ are extended to $f(X)$ and the same happens for $Y$.

$\endgroup$
1
  • 6
    $\begingroup$ +1. Thank you for this contribution, which so clearly focuses on the essential idea. Welcome to our site! $\endgroup$
    – whuber
    Commented Sep 20, 2015 at 13:26
11
$\begingroup$

Yes, $g(X)$ and $h(Y)$ are independent for any functions $g$ and $h$ so long as $X$ and $Y$ are independent. It's a very well known results, which is studied in probability theory courses. I'm sure you can find it in any standard text like Billingsley's.

$\endgroup$
3
  • $\begingroup$ Thanks, I am currently studying Hogg & Craig and MGB. Billingsley is the next logical step. $\endgroup$
    – JohnK
    Commented Apr 23, 2014 at 15:02
  • 3
    $\begingroup$ Billinglsey's a torture unless you're mathematician and already studied measures. Partarathy's intro is much easier 2-in-1 book, Alan Karr's Probability text is also easy read. $\endgroup$
    – Aksakal
    Commented Apr 23, 2014 at 15:28
  • $\begingroup$ Another easier text than Billingsley's: probability.ca/jeff/grprobbook.html $\endgroup$
    – Adrian
    Commented Jun 15, 2019 at 1:50
8
$\begingroup$

Not as an alternative, but as an addition, to the previous brilliant answers: the independence of functions of independent random variables is, in fact, quite intuitive.

Usually, we think that $X$ and $Y$ being independent means that knowing the value of $X$ gives no information about the value of $Y$ and vice versa. This interpretation obviously implies that you can't somehow "squeeze" any information out by applying a function (or by any other means actually).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.