Zero correlation of all functions of random variables implying independence

Question

Independence between random variables $X$ and $Y$ implies that $\text{Corr}\left(f(X),g(Y)\right)=0$ for arbitrary functions $f(\cdot)$ and $g(\cdot)$ (here is a related thread).

But is the following statement, or a similar one (perhaps more rigorously defined), correct?

If $\text{Corr}\left(f(X),g(Y)\right)=0$ for all possible functions $f(\cdot)$ and $g(\cdot)$, then $X$ and $Y$ are independent.

Answer 1

Using indicator functions of measurable sets like$$f(x)=\mathbb I_A(x)\quad g(x)=\mathbb I_B(x)$$leads to$$\text{cov}(f(X),g(Y))=\mathbb P(X\in A,Y\in B)-\mathbb P(X\in A)\mathbb P(Y\in B)$$therefore implying independence. As shown in the following snapshot of A. Dembo's probability course, proving the result for indicator functions is enough.

This is due to this monotone class theorem:

Answer 2

@Xi'an gives probably the simplest set of functions $f,\,g$ that will work. Here's a more general argument:

It is sufficient to show that the characteristic function $E[\exp(itX+isY)]$ factors into $E[\exp(itX)]E[\exp(iSY)]$, because characteristic functions determine distributions.

Therefore, it is sufficient to show zero correlation

• when $f,\,g$ are of the form $f_t(x)=\exp(itx)$ and $f_s(y)=\exp(isy)$
• so $\sin(tx)$ and $\cos(sy)$ are also sufficient
• by the Weierstrass approximation theorem, the sines and cosines can be approximated by polynomials, which also suffice
• more generally, by the Stone-Weierstrass theorem, any other set of continuous functions closed under addition and multiplication, containing the constants, and separating points will also do ['separates points' means for any $x_1$ and $x_2$ you can find $f$ so that $f(x_1)\neq f(x_2)$, and similarly for $y$ and $g$]
• the construction of integrals from indicator functions shows you can also use constant functions as @Xi'an does
• and, like, wavelets or whatever

It might occasionally be useful to note that you don't have to use the same set of functions for $f$ as for $g$. For example, you could use indicator functions for $f$ and polynomials for $g$ if that somehow made your life easier

Answer 3

Any continuous random variable can be mapped into a uniform [0,1] random variable using the cumulative distribution function. If the variables are independent, then the joint distribution on the 1x1 square will be the product of the two uniform margins and so uniform too. For the variables to be dependent, the joint distribution is not equal to the product, and therefore not uniform. The 1x1 square has bumps and dips in it. We then apply a permutation of intervals/blocks along each axis to rearrange those bumps along the diagonal and the dips far away from it - like permuting the rows and columns of a matrix with the Cuthill-McKee algorithm. This makes the correlation non-zero. Thus, zero correlation for all functions of continuous random variables implies independence.

Answer 4

If $\text{Corr}\left(f(X),g(Y)\right)=0$ for all possible functions $f(\cdot)$ and $g(\cdot)$, then $X$ and $Y$ are independent.

In the ref that I have the opposite is affirmed. If $X$ and $Y$ are independent we have that:

$E[f(X)]E[g(Y)]-E[f(X)g(Y)]=0$ (then $corr[f(X),g(Y)]=0$)

for any $f()$ and $g()$.

In words, we have no chance to find dependencies. Indeed if exist, them must be revealed by some functional relations. See: Econometrics – Verbeek; 5th edition pag 463. But some distributions/moments/functions conditions seems me implicit.

To move in the opposite direction is permitted, so from $\text{Corr}\left(f(X),g(Y)\right)=0$ the independence is implied.

However can be useful to note that the condition $\text{Corr}\left(f(X),g(Y)\right)=0$ imply some restrictions on the distributions/functions/moments. In some cases, this condition can fail. For example if $X$ and $Y$ are independent Cauchy r.vs: $\text{Corr}\left(f(X),g(Y)\right)=0$ not hold, or at lest not for some $f()$ and $g()$. Then, the condition in argument and the independence are not completely equivalent.

Answer 5

Two variables being dependent means that there is some value(s) of one variable that make some value(s) of the other variable more likely (the general statement is that it changes the probability, but WLOG we can assume that it increases the probability). And if that is the cases, then clearly there is positive correlation between the first variable having the value(s) in question, and the second variable having the value(s) in question. This correlation can be reflected in correlation between functions by taking functions that have different outputs depending on whether the variables take on the value(s) in question.

As a practical matter, this isn't generally a good method of proving independence. Given any countable set of functions, it's possible to construct two dependent variables for which all those functions are uncorrelated. So you have to prove that an uncountable set of functions are uncorrelated, at which point it's probably easier to just prove independence directly.