Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

For 2 variables to be independent of each other, should the correlation = 0 or mutual information = 0 or covariance = 0. I have seen different conditions and all these are really confusing.

share|improve this question
    
What different conditions have you seen? –  Glen_b Jul 30 at 11:52

2 Answers 2

up vote 6 down vote accepted

1) covariance=0 implies correlation=0 (unless both the variances are also 0 in which case correlation is 0/0).

2) correlation (or covariance 0) is a necessary but not sufficient for independence. Independence implies both correlation and covariance are 0, but both can be 0 with perfectly dependent data.

See here:

If the variables are independent, Pearson's correlation coefficient is 0, but the converse is not true because the correlation coefficient detects only linear dependencies between two variables. For example, suppose the random variable X is symmetrically distributed about zero, and $Y = X^2$. Then Y is completely determined by X, so that X and Y are perfectly dependent, but their correlation is zero

3) As it says here:

I(X; Y) = 0 if and only if X and Y are independent random variables.

That is, mutual information 0 implies independence (and vice versa).

share|improve this answer

Two variables are considered independent if they are orthogonal to each other. Which means that their dot product is equal to 0.

$\mathbf{a}\cdot \mathbf{b} = \sum_{i=1}^n a_ib_i = a_1b_1 + a_2b_2 + \cdots + a_nb_n = 0$

In other words one variable doesn't contain any information on the second variable.

Different conditions for independence come from different scientific fields like linear algebra or probability theory, but the underlying concept is always the same.

share|improve this answer
3  
"the underlying concept is always the same"? For random variables $X$ and $Y$, independence has nothing to do with orthogonality. $X\cdot Y$ may not even make sense. –  Stefan Hansen Jul 30 at 12:57
    
It makes sense to define the dot product between two random variables as the expectation over the product. However, also in that case it is only related to the covariance and therefore only indirectly with independence. –  fabee Jul 30 at 17:24

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.