23
$\begingroup$

Independence between random variables $X$ and $Y$ implies that $\text{Corr}\left(f(X),g(Y)\right)=0$ for arbitrary functions $f(\cdot)$ and $g(\cdot)$ (here is a related thread).

But is the following statement, or a similar one (perhaps more rigorously defined), correct?

If $\text{Corr}\left(f(X),g(Y)\right)=0$ for all possible functions $f(\cdot)$ and $g(\cdot)$, then $X$ and $Y$ are independent.

Improve this question
$\endgroup$
2
  • 3
    $\begingroup$ I would say all bounded continuous functions rather than all possible functions. For some functions the variance may be infinite, so correlations will not be defined. But for bounded continuous functions that problem doesn't arise and that's a big enough class of functions to get the result. $\endgroup$
    – Michael Hardy
    Jan 9, 2021 at 20:32
  • $\begingroup$ See stats.stackexchange.com/a/638988/69508 $\endgroup$
    – Galen
    Feb 10 at 6:24

6 Answers 6

Reset to default
34
$\begingroup$

Using indicator functions of measurable sets like$$f(x)=\mathbb I_A(x)\quad g(x)=\mathbb I_B(x)$$leads to$$\text{cov}(f(X),g(Y))=\mathbb P(X\in A,Y\in B)-\mathbb P(X\in A)\mathbb P(Y\in B)$$therefore implying independence. As shown in the following snapshot of A. Dembo's probability course, proving the result for indicator functions is enough.

enter image description here

This is due to this monotone class theorem:

enter image description here

Improve this answer
$\endgroup$
5
  • 2
    $\begingroup$ (+1) I was thinking of attacking moments, but this is clearly the simplest approach. $\endgroup$
    – gunes
    Jan 7, 2021 at 9:06
  • $\begingroup$ How does this prove the hypothesis? I think there are missing steps. $\endgroup$
    – Galen
    Feb 13, 2021 at 17:16
  • 1
    $\begingroup$ It isn't explicit how these indicator functions account for all functions, but rather appear to be a choice of functions. Can you clarify how this proof meets this general criterion for the hypothesis? $\endgroup$
    – Galen
    Feb 13, 2021 at 18:21
  • 1
    $\begingroup$ Excellent, thank you. I think I need to brush up on measure theory, but I get the gist now. $\endgroup$
    – Galen
    Feb 13, 2021 at 18:28
  • 1
    $\begingroup$ This theorem of Halmos s incredibly powerful! $\endgroup$
    – Xi'an
    Feb 13, 2021 at 18:31
7
$\begingroup$

@Xi'an gives probably the simplest set of functions $f,\,g$ that will work. Here's a more general argument:

It is sufficient to show that the characteristic function $E[\exp(itX+iSY)]$ factors into $E[\exp(itX)]E[\exp(iSY)]$, because characteristic functions determine distributions.

Therefore, it is sufficient to show zero correlation

  • when $f,\,g$ are of the form $f_t(x)=\exp(itx)$ and $f_s(y)=\exp(isy)$
  • so $\sin(tx)$ and $\cos(sy)$ are also sufficient
  • by the Weierstrass approximation theorem, the sines and cosines can be approximated by polynomials, which also suffice
  • more generally, by the Stone-Weierstrass theorem, any other set of continuous functions closed under addition and multiplication, containing the constants, and separating points will also do ['separates points' means for any $x_1$ and $x_2$ you can find $f$ so that $f(x_1)\neq f(x_2)$, and similarly for $y$ and $g$]
  • the construction of integrals from indicator functions shows you can also use constant functions as @Xi'an does
  • and, like, wavelets or whatever

It might occasionally be useful to note that you don't have to use the same set of functions for $f$ as for $g$. For example, you could use indicator functions for $f$ and polynomials for $g$ if that somehow made your life easier

Improve this answer
$\endgroup$
0
5
$\begingroup$

Any continuous random variable can be mapped into a uniform [0,1] random variable using the cumulative distribution function. If the variables are independent, then the joint distribution on the 1x1 square will be the product of the two uniform margins and so uniform too. For the variables to be dependent, the joint distribution is not equal to the product, and therefore not uniform. The 1x1 square has bumps and dips in it. We then apply a permutation of intervals/blocks along each axis to rearrange those bumps along the diagonal and the dips far away from it - like permuting the rows and columns of a matrix with the Cuthill-McKee algorithm. This makes the correlation non-zero. Thus, zero correlation for all functions of continuous random variables implies independence.

Improve this answer
$\endgroup$
2
  • 2
    $\begingroup$ Interesting, thank you! $\endgroup$
    – Richard Hardy
    Jan 8, 2021 at 7:40
  • 1
    $\begingroup$ Could the downvoter provide a comment? $\endgroup$
    – Richard Hardy
    Jan 8, 2021 at 18:26
0
$\begingroup$

If $\text{Corr}\left(f(X),g(Y)\right)=0$ for all possible functions $f(\cdot)$ and $g(\cdot)$, then $X$ and $Y$ are independent.

In the ref that I have the opposite is affirmed. If $X$ and $Y$ are independent we have that:

$E[f(X)]E[g(Y)]-E[f(X)g(Y)]=0$ (then $corr[f(X),g(Y)]=0$)

for any $f()$ and $g()$.

In words, we have no chance to find dependencies. Indeed if exist, them must be revealed by some functional relations. See: Econometrics – Verbeek; 5th edition pag 463. But some distributions/moments/functions conditions seems me implicit.

To move in the opposite direction is permitted, so from $\text{Corr}\left(f(X),g(Y)\right)=0$ the independence is implied.

However can be useful to note that the condition $\text{Corr}\left(f(X),g(Y)\right)=0$ imply some restrictions on the distributions/functions/moments. In some cases, this condition can fail. For example if $X$ and $Y$ are independent Cauchy r.vs: $\text{Corr}\left(f(X),g(Y)\right)=0$ not hold, or at lest not for some $f()$ and $g()$. Then, the condition in argument and the independence are not completely equivalent.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Just to clarify: the implication from independence to lack of correlation is not my focus; I am only asking about the case of implication from lack of correlation to independence. Thus I think only your last two paragraphs are really of interest, especially If we want move in the opposite direction, I fear that with some peculiar distributions and/or transformations some problems can appear. But in general the idea hold. I think Xi'an's answer addresses your fear, does it not? $\endgroup$
    – Richard Hardy
    Jan 7, 2021 at 15:52
  • 1
    $\begingroup$ I edited my answer spending more attention on some details. $\endgroup$
    – markowitz
    Jan 7, 2021 at 16:58
  • 1
    $\begingroup$ The example with Cauchy random variables is helpful. $\endgroup$
    – Richard Hardy
    Jan 8, 2021 at 18:28
0
$\begingroup$

Two variables being dependent means that there is some value(s) of one variable that make some value(s) of the other variable more likely (the general statement is that it changes the probability, but WLOG we can assume that it increases the probability). And if that is the cases, then clearly there is positive correlation between the first variable having the value(s) in question, and the second variable having the value(s) in question. This correlation can be reflected in correlation between functions by taking functions that have different outputs depending on whether the variables take on the value(s) in question.

As a practical matter, this isn't generally a good method of proving independence. Given any countable set of functions, it's possible to construct two dependent variables for which all those functions are uncorrelated. So you have to prove that an uncountable set of functions are uncorrelated, at which point it's probably easier to just prove independence directly.

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ This characterization looks like a great way to prove two variables are not independent, though. $\endgroup$
    – whuber ♦
    Jan 9, 2021 at 18:03
  • $\begingroup$ Thank you for the answer. Could you perhaps digest it to a more direct answer to my original question? $\endgroup$
    – Richard Hardy
    Jan 9, 2021 at 23:17
  • $\begingroup$ Is that second comment true? Since $Q := \{(-\infty, q] : q \in \mathbb Q\}$ is a $\pi$-system that generates the Borel $\sigma$-algebra, if $\mathbf 1_{A}(X)$ is uncorrelated with $\mathbf 1_B(Y)$ for every $A,B \in Q$ then $X \perp Y$, and that's a countable collection of indicators $\endgroup$
    – jld
    Jan 11, 2021 at 14:59
-4
$\begingroup$

Correlation catches only the linear dependence between two variables.

A and B are dependent but uncorrelated if $A = B^2$ for example

Pure independence implies the stochastic independence, which is that the occurrence of one does not affect the occurrence of the other. Similarly, two random variables are independent if the realization of one does not affect the probability distribution of the other (copied from the wiki)

Improve this answer
$\endgroup$
6
  • 3
    $\begingroup$ Thanks for the general definition. How does that answer my question? Can we establish independence between $X$ and $Y$ when we know that $\text{Corr}\left(f(X),g(Y)\right)=0$ for all possible functions $f(\cdot)$ and $g(\cdot)$? The functions are of course not limited to just linear ones. $\endgroup$
    – Richard Hardy
    Jan 7, 2021 at 8:54
  • $\begingroup$ No we can not. Even though the correlation is zero, variables might be statistically dependent $\endgroup$
    – Emil Mirzayev
    Jan 7, 2021 at 9:24
  • 3
    $\begingroup$ I think that if Xi'an's answer is correct, your comment must be wrong. Would you challenge his answer or concede? $\endgroup$
    – Richard Hardy
    Jan 7, 2021 at 9:50
  • 2
    $\begingroup$ @EmilMirzayev Please don't perpetuate this fallacy about $A,B$ being dependent but uncorrelated if $A=B^2$. We have $Cov(A,B) = E(B^3) - E(B^2)E(B)$ and in general this is not zero. Try an Exponential with parameter $1$, whose moments are $E(B^n) = n!$. $\endgroup$
    – Alecos Papadopoulos
    Jan 8, 2021 at 4:24
  • $\begingroup$ @AlecosPapadopoulos Well, if the domain is symmetrical (e.g. [-1.1]), then they will be uncorrelated. But following Xi'an's answer, we can take indicator functions for a nonsymmetric set, revealing nonzero correlation. $\endgroup$
    – Acccumulation
    Jan 9, 2021 at 9:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.