Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Can we say anything about the dependence of a random variable and a function of a random variable? For example is $X^2$ dependent on $X$?

share|improve this question
5  
If $X$ and $f(X)$ are independent, then $f(X)$ is constant almost surely. That is, there exists $a$ such that $\mathbb P(f(X) = a) = 1$. –  cardinal Oct 1 '11 at 15:03
2  
@cardinal -why not make that an answer? –  Karl Oct 1 '11 at 16:30
    
@cardinal, I wanted to ask John to elaborate on his comment. I took it for granted that the function considered was a given, deterministic function. In the process, I ended up writing out an argument for the result you stated instead. Any comments are most welcome and appreciated. –  NRH Oct 1 '11 at 22:45
    
Yes, $X^2$ is dependent on $X$, since if you know $X$ then you know $X^2$. $X$ and $Y$ are only independent if knowledge of the value $X$ did not affect your knowledge of the distribution of $Y$. –  Henry Oct 1 '11 at 23:21
1  
@iamrohitbanga: If $X \in \{-1,1\}$ then $X^2 = 1$ almost surely. So, $X$ is independent of $X^2$ in this very particular case. –  cardinal Oct 12 '11 at 1:28
show 4 more comments

2 Answers 2

up vote 13 down vote accepted

Here is a proof of @cardinal's comment with a small twist. If $X$ and $f(X)$ are independent then $$ \begin{array}{lcl} P(X \in A \cap f^{-1}(B)) & = & P(X \in A, f(X) \in B) \\ & = & P(X \in A) P(f(X) \in B) \\ & = & P(X \in A) P(X \in f^{-1}(B)) \end{array} $$ Taking $A = f^{-1}(B)$ yields the equation $$P(f(X) \in B) = P(f(X) \in B)^2,$$ which has the two solutions 0 and 1. Thus $P(f(X) \in B) \in \{0, 1\}$ for all $B$. In complete generality, it's not possible to say more. If $X$ and $f(X)$ are independent, then $f(X)$ is a variable such that for any $B$ it is either in $B$ or in $B^c$ with probability 1. To say more, one needs more assumptions, e.g. that singleton sets $\{b\}$ are measurable.

However, details at the measure theoretic level do not seem to be the main concern of the OP. If $X$ is real and $f$ is a real function (and we use the Borel $\sigma$-algebra, say), then taking $B = (-\infty, b]$ it follows that the distribution function for the distribution of $f(X)$ only takes the values 0 and 1, hence there is a $b$ at which it jumps from $0$ to $1$ and $P(f(X) = b) = 1$.

At the end of the day, the answer to the OPs question is that $X$ and $f(X)$ are generally dependent and only independent under very special circumstances. Moreover, the Dirac measure $\delta_{f(x)}$ always qualifies for a conditional distribution of $f(X)$ given $X = x$, which is a formal way of saying that knowing $X = x$ then you also know exactly what $f(X)$ is. This special form of dependence with a degenerate conditional distribution is characteristic for functions of random variables.

share|improve this answer
    
(+1) Sorry. As I was composing my answer, I didn't get an update that you'd submitted one as well. :) –  cardinal Oct 1 '11 at 22:44
add comment

Lemma: Let $X$ be a random variable and let $f$ be a (Borel measurable) function such that $X$ and $f(X)$ are independent. Then $f(X)$ is constant almost surely. That is, there is some $a \in \mathbb R$ such that $\mathbb P(f(X) = a) = 1$.

The proof is below; but, first, some remarks. The Borel measurability is just a technical condition to ensure that we can assign probabilities in a reasonable and consistent way. The "almost surely" statement is also just a technicality.

The essence of the lemma is that if we want $X$ and $f(X)$ to be independent, then our only candidates are functions of the form $f(x) = a$.

Contrast this with the case of functions $f$ such that $X$ and $f(X)$ are uncorrelated. This is a much, much weaker condition. Indeed, consider any random variable $X$ with mean zero, finite absolute third moment and that is symmetric about zero. Take $f(x) = x^2$, as in the example in the question. Then $\mathrm{Cov}(X,f(X)) = \mathbb E Xf(X) = \mathbb E X^3 = 0$, so $X$ and $f(X) = X^2$ are uncorrelated.

Below, I give the simplest proof I could come up with for the lemma. I've made it exceedingly verbose so that all the details are as obvious as possible. If anyone sees ways to improve it or simplify it, I'd enjoy knowing.

Idea of proof: Intuitively, if we know $X$, then we know $f(X)$. So, we need to find some event in $\sigma(X)$, the sigma algebra generated by $X$, that relates our knowledge of $X$ to that of $f(X)$. Then, we use that information in conjunction with the assumed independence of $X$ and $f(X)$ to show that our available choices for $f$ have been severely constrained.

Proof of lemma: Recall that $X$ and $Y$ are independent if and only if for all $A \in \sigma(X)$ and $B \in \sigma(Y)$, $\renewcommand{\Pr}{\mathbb P}\Pr(X \in A, Y \in B) = \Pr(X \in A) \Pr(Y \in B)$. Let $Y = f(X)$ for some Borel measurable function $f$ such that $X$ and $Y$ are independent. Define $\newcommand{\o}{\omega}A(y) = \{\o: f(X(\o)) \leq y\}$. Then, $$ A(y) = \{\o: X(\o) \in f^{-1}((-\infty,y])\} $$ and since $(-\infty,y]$ is a Borel set and $f$ is Borel-measurable, then $f^{-1}((-\infty,y])$ is also a Borel set. This implies that $A(y) \in \sigma(X)$ (by definition(!) of $\sigma(X)$).

Since $X$ and $Y$ are assumed independent and $A(y) \in \sigma(X)$, then $$ \Pr(X \in A(y), Y \leq y) = \Pr(X \in A(y)) \Pr(Y \leq y) = \Pr(f(X) \leq y) \Pr(f(X) \leq y) \>, $$ and this holds for all $y \in \mathbb R$. But, by definition of $A(y)$ $$ \Pr(X \in A(y), Y \leq y) = \Pr(f(X) \leq y, Y \leq y) = \Pr(f(X) \leq y) \> . $$ Combining these last two, we get that for every $y \in \mathbb R$, $$ \Pr(f(X) \leq y) = \Pr(f(X) \leq y) \Pr(f(X) \leq y) \>, $$ so $\Pr(f(X) \leq y) = 0$ or $\Pr(f(X) \leq y) = 1$. This means there must be some constant $a \in \mathbb R$ such that the distribution function of $f(X)$ jumps from zero to one at $a$. In other words, $f(X) = a$ almost surely.

NB: Note that the converse is also true by an even simpler argument. That is, if $f(X) = a$ almost surely, then $X$ and $f(X)$ are independent.

share|improve this answer
    
+1, I should be sorry - to steal your argument is not very polite. It's great that you spelled out the difference between independence and being uncorrelated in this context. –  NRH Oct 1 '11 at 22:55
    
No "stealing" involved, nor impoliteness. :) Though many of the ideas and comments are similar (as you'd expect for a question like this!), I think the two posts are nicely complementary. In particular, I like how at the beginning of your post you didn't constrain yourself to real-valued random variables. –  cardinal Oct 1 '11 at 23:01
    
@NRH accepting your answer as the initial part of your proof seems easier to grasp for a novice like me. Nevertheless +1 cardinal for your answer. –  Rohit Banga Oct 2 '11 at 16:02
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.