8
$\begingroup$

I was confused by what it means when a Random variable is a deterministic function of another Random variable yet is independent of it? How is this possible?

Here's the question: Consider three random variables $X,Y$ and $Z$ such that $Z$ is independent of $X$ and of $Y$ and is a deterministic function of $(X,Y)$. Is $H(Z)=0$? If so prove it; if not, give a counter example.

$\endgroup$
9
  • 3
    $\begingroup$ Can you provide reference mentioning it and a broader context? Basically, if one random variable is a function of another one it is dependent of it (stats.stackexchange.com/questions/16321/… and stats.stackexchange.com/questions/231425/… ) $\endgroup$
    – Tim
    Commented Sep 18, 2016 at 17:53
  • $\begingroup$ @Tim That's what i thought too . Here's the question . Consider three random variables X,Y and Z such that Z is independent of X and of Y and is a deterministic function of (X,Y). Is H(Z)=0 ? If so prove it ; if not , give a counter example $\endgroup$ Commented Sep 18, 2016 at 17:59
  • $\begingroup$ please edit to add those details. Btw is it a homework? $\endgroup$
    – Tim
    Commented Sep 18, 2016 at 18:38
  • 1
    $\begingroup$ Do you mean Z=H(X,Y)=0 or does H(Z) mean something else? $\endgroup$
    – Patty
    Commented Sep 18, 2016 at 19:01
  • 2
    $\begingroup$ @Christopher still I think you should add [self-study] tag and check: stats.stackexchange.com/tags/self-study/info $\endgroup$
    – Tim
    Commented Sep 18, 2016 at 19:10

2 Answers 2

9
$\begingroup$

A non trivial univariate real random variable that is a deterministic function of another random variable is not independent of it (see Mark L. Stone's answer for an example with a constant random variable). However, when more than two random variables are involved independence shows counterintuitive behaviours.

I'll give an example of $Z$ deterministic function of $X$ and $Y$ but independent from $X$ and $Y$.

Let $X$ and $Y$ be independent Bernouilli variables with $p=0.5$ (for example, $X$ and $Y$ are the results of tossing a coin each).

Let $f(X,Y)$ equal $1$ if $X=Y$ and $0$ if $X$ is different from $Y$.

Let $Z=f(X,Y)$.

You can easily see that $P(X=0)=0.5=P(X=0\mid Z=1)=P(X=0\mid Z=0)$ and that $P(X=1)=0.5=P(X=1\mid Z=1)=P(X=1\mid Z=0)$, proving that $X$ and $Z$ are independent, or using another definition of independence:

\begin{align*}P(x=0\text{ and }Z=0) &= P(X=0\text{ and }X \text{ different from }Y) \\ &= P(X=0\text{ and }Y=1) \\ &= 0.5 \cdot 0.5 = 0.25 \\\\ P(X=0) \cdot P(Z=0) &= 0.5 \cdot 0.5 = 0.25 \end{align*}

The same operation can be done for all values of $X$ and $Z$, thus proving that $P(X=a\text{ and }Z=b) = P(X=a)\cdot P(Z=b)$ for every value of $a$ and $b$.

Furthermore, the same proof holds for $Y$, therefore proving that $Y$ and $Z$ are independent and $X$ and $Z$ are independent. In fact, $X$, $Y$ and $Z$ are pairwise independent while $X$, $Y$ and $Z$ are not independent considered as a whole (not jointly independent). Interestingly, $Z$ is not independent of $(X, Y)$.

$\endgroup$
5
  • 1
    $\begingroup$ Your statement "An univariate real random variable that is a deterministic function of another random variable is not independent of it. " is incorrect, as shown in my answer. $\endgroup$ Commented Sep 18, 2016 at 19:32
  • $\begingroup$ @Patty , the counter example i provided was an XOR . Seems like this is correct . Yes , Pere . +1 (Pairwise independence does not imply joint independence ) . $\endgroup$ Commented Sep 18, 2016 at 19:34
  • $\begingroup$ Well, the correctness of your edited statement depends on your definition of non-trivial function. Consider any function $f(x)$ which is one to one. Apply it to a random variable $Y$ which is a constant with probability one. Then $f(Y)$ is independent of $Y$. $\endgroup$ Commented Sep 18, 2016 at 19:39
  • $\begingroup$ Yes, I edited it again to apply "trivial" to the resulting random variable and not to the function. I'm not sure if there is a rigorous definition of trivial random variable, but I think it's clear enough - in fact, I'm not sure if there is a definition of random variable that includes something that always gives the same result (a deterministic random variable). However, since it points to your answer I think readers will easily understand what is the exception about. $\endgroup$
    – Pere
    Commented Sep 18, 2016 at 19:44
  • $\begingroup$ @Pere I took the liberty of adding MathJax to your (great) answer to improve readability. I hope that is okay - otherwise reject the edit! $\endgroup$
    – Therkel
    Commented Sep 19, 2016 at 7:12
6
$\begingroup$

A random variable which is a constant with probability 1 is independent of itself. I leave the trivial proof to you as an exercise.

Consider the deterministic function f(x) = x, applied to the random variable $Y$, which equals a constant, say $e^\pi$, with probability one. Therefore, the random variable $f(Y)$ is independent of $Y$. If you don't like that example because $f(Y)$ is the same as $Y$, then use the function $f(x) = 2x$. Same conclusion.

This provides a counterexample to the incorrect statement "An univariate real random variable that is a deterministic function of another random variable is not independent of it." in the first paragraph by the answer by @Pere .

$\endgroup$
3
  • $\begingroup$ True , the link provided by @Tim serves as a good example of countering that $\endgroup$ Commented Sep 18, 2016 at 19:34
  • 1
    $\begingroup$ @Mark L. Stone: I edited my answer to exclude the trivial case of a non random random variable. $\endgroup$
    – Pere
    Commented Sep 18, 2016 at 19:39
  • 1
    $\begingroup$ This answer would be more interesting (rather than just pedantic) if you went on to describe the technical conditions (for "non-triviality") needed to make the statement true. $\endgroup$ Commented Sep 19, 2016 at 3:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.