What are the properties for independence For 2 variables to be independent of each other, should the correlation = 0 or mutual information = 0 or covariance = 0. I have seen different conditions and all these are really confusing.
 A: *

*covariance=0 implies correlation=0 (as long as the variances aren't 0).


*correlation (or covariance 0) is necessary but not sufficient for independence. Independence implies both correlation and covariance are 0, but both can be 0 with perfectly dependent data.
See here:

If the variables are independent, Pearson's correlation coefficient is 0, but the converse is not true because the correlation coefficient detects only linear dependencies between two variables. For example, suppose the random variable X is symmetrically distributed about zero, and $Y = X^2$. Then Y is completely determined by X, so that X and Y are perfectly dependent, but their correlation is zero

As Dilip says, we need $E[|X^3|]$ to be finite for the specific example here; there will be similar criteria in other cases. For example, if $Y=f(X)$ for some even $f$, we'd need $E[|X.f(X)|]$ to be finite for the symmetry to make the covariance 0.


*As it says here:


I(X; Y) = 0 if and only if X and Y are independent random variables.

That is, mutual information 0 implies independence (and vice versa).
A: Two variables are considered independent if they are orthogonal to each other. Which means that their dot product is equal to 0.
$\mathbf{a}\cdot \mathbf{b} = \sum_{i=1}^n a_ib_i = a_1b_1 + a_2b_2 + \cdots + a_nb_n = 0$
In other words one variable doesn't contain any information on the second variable.
Different conditions for independence come from different scientific fields like linear algebra or probability theory, but the underlying concept is always the same.
