5
$\begingroup$

I was confused by what it means when a Random variable is a deterministic function of another Random variable yet is independent of it? How is this possible?

Here's the question: Consider three random variables $X,Y$ and $Z$ such that $Z$ is independent of $X$ and of $Y$ and is a deterministic function of $(X,Y)$. Is $H(Z)=0$? If so prove it; if not, give a counter example.

$\endgroup$
  • 3
    $\begingroup$ Can you provide reference mentioning it and a broader context? Basically, if one random variable is a function of another one it is dependent of it (stats.stackexchange.com/questions/16321/… and stats.stackexchange.com/questions/231425/… ) $\endgroup$ – Tim Sep 18 '16 at 17:53
  • $\begingroup$ @Tim That's what i thought too . Here's the question . Consider three random variables X,Y and Z such that Z is independent of X and of Y and is a deterministic function of (X,Y). Is H(Z)=0 ? If so prove it ; if not , give a counter example $\endgroup$ – Christopher Sep 18 '16 at 17:59
  • $\begingroup$ please edit to add those details. Btw is it a homework? $\endgroup$ – Tim Sep 18 '16 at 18:38
  • 1
    $\begingroup$ Do you mean Z=H(X,Y)=0 or does H(Z) mean something else? $\endgroup$ – Patty Sep 18 '16 at 19:01
  • 2
    $\begingroup$ @Christopher still I think you should add [self-study] tag and check: stats.stackexchange.com/tags/self-study/info $\endgroup$ – Tim Sep 18 '16 at 19:10
9
$\begingroup$

A non trivial univariate real random variable that is a deterministic function of another random variable is not independent of it (see Mark L. Stone's answer for an example with a constant random variable). However, when more than two random variables are involved independence shows counterintuitive behaviours.

I'll give an example of $Z$ deterministic function of $X$ and $Y$ but independent from $X$ and $Y$.

Let $X$ and $Y$ be independent Bernouilli variables with $p=0.5$ (for example, $X$ and $Y$ are the results of tossing a coin each).

Let $f(X,Y)$ equal $1$ if $X=Y$ and $0$ if $X$ is different from $Y$.

Let $Z=f(X,Y)$.

You can easily see that $P(X=0)=0.5=P(X=0\mid Z=1)=P(X=0\mid Z=0)$ and that $P(X=1)=0.5=P(X=1\mid Z=1)=P(X=1\mid Z=0)$, proving that $X$ and $Z$ are independent, or using another definition of independence:

\begin{align*}P(x=0\text{ and }Z=0) &= P(X=0\text{ and }X \text{ different from }Y) \\ &= P(X=0\text{ and }Y=1) \\ &= 0.5 \cdot 0.5 = 0.25 \\\\ P(X=0) \cdot P(Z=0) &= 0.5 \cdot 0.5 = 0.25 \end{align*}

The same operation can be done for all values of $X$ and $Z$, thus proving that $P(X=a\text{ and }Z=b) = P(X=a)\cdot P(Z=b)$ for every value of $a$ and $b$.

Furthermore, the same proof holds for $Y$, therefore proving that $Y$ and $Z$ are independent and $X$ and $Z$ are independent. In fact, $X$, $Y$ and $Z$ are pairwise independent while $X$, $Y$ and $Z$ are not independent considered as a whole (not jointly independent). Interestingly, $Z$ is not independent of $(X, Y)$.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ Your statement "An univariate real random variable that is a deterministic function of another random variable is not independent of it. " is incorrect, as shown in my answer. $\endgroup$ – Mark L. Stone Sep 18 '16 at 19:32
  • $\begingroup$ @Patty , the counter example i provided was an XOR . Seems like this is correct . Yes , Pere . +1 (Pairwise independence does not imply joint independence ) . $\endgroup$ – Christopher Sep 18 '16 at 19:34
  • $\begingroup$ Well, the correctness of your edited statement depends on your definition of non-trivial function. Consider any function $f(x)$ which is one to one. Apply it to a random variable $Y$ which is a constant with probability one. Then $f(Y)$ is independent of $Y$. $\endgroup$ – Mark L. Stone Sep 18 '16 at 19:39
  • $\begingroup$ Yes, I edited it again to apply "trivial" to the resulting random variable and not to the function. I'm not sure if there is a rigorous definition of trivial random variable, but I think it's clear enough - in fact, I'm not sure if there is a definition of random variable that includes something that always gives the same result (a deterministic random variable). However, since it points to your answer I think readers will easily understand what is the exception about. $\endgroup$ – Pere Sep 18 '16 at 19:44
  • $\begingroup$ @Pere I took the liberty of adding MathJax to your (great) answer to improve readability. I hope that is okay - otherwise reject the edit! $\endgroup$ – Therkel Sep 19 '16 at 7:12
6
$\begingroup$

A random variable which is a constant with probability 1 is independent of itself. I leave the trivial proof to you as an exercise.

Consider the deterministic function f(x) = x, applied to the random variable $Y$, which equals a constant, say $e^\pi$, with probability one. Therefore, the random variable $f(Y)$ is independent of $Y$. If you don't like that example because $f(Y)$ is the same as $Y$, then use the function $f(x) = 2x$. Same conclusion.

This provides a counterexample to the incorrect statement "An univariate real random variable that is a deterministic function of another random variable is not independent of it." in the first paragraph by the answer by @Pere .

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ True , the link provided by @Tim serves as a good example of countering that $\endgroup$ – Christopher Sep 18 '16 at 19:34
  • 1
    $\begingroup$ @Mark L. Stone: I edited my answer to exclude the trivial case of a non random random variable. $\endgroup$ – Pere Sep 18 '16 at 19:39
  • 1
    $\begingroup$ This answer would be more interesting (rather than just pedantic) if you went on to describe the technical conditions (for "non-triviality") needed to make the statement true. $\endgroup$ – R.. GitHub STOP HELPING ICE Sep 19 '16 at 3:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.