I know what it means to say two variables are independent, but can't understand what does it mean to say two variables are orthogonal.Can anyone help me?
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
4
In order to speak about orthogonality you need to define an inner product first.
If we consider random variables with finite second moment, covariance can be shown to be an inner product. In this case two random variables are orthogonal if and only if they are uncorrelated:
$$ 0 = \text{cov}(X, Y) = \mathbb{E}[X Y] - \mathbb{E}[X] \mathbb{E}[Y] $$
Note that zero covariance does not imply independence (in general). For details, see Covariance and independence?
Also, one could have defined inner product differently, which would lead to another notion of orthogonality. Yet, the one I described, seems more common to me.
-
$\begingroup$ Sorry, I don't understand. The dot product is sum(X_i*Y_i), but how does this relates to E[XY]-E[X]E[Y]? $\endgroup$– Mario GSCommented May 3, 2017 at 19:10
-
$\begingroup$ @MarioGS,
sum(X_i Y_i)is a special case of dot product for Euclidean vector spaces. However, there are other kinds of vector spaces (space of random variables with finite 2nd moment is not Euclidean), often equipped with a notion of a dot product $\endgroup$ Commented May 4, 2017 at 22:25 -
$\begingroup$ In that case, if X and Y are random vectors, their dot product is not defined by the euclidian definition of dot product such as X.Y=Sum(Xi*Yi), but actually as X.Y=E[X,Y]-E[X]E[Y], is that right? $\endgroup$– Mario GSCommented May 10, 2017 at 10:56
-
$\begingroup$ @MarioGS, yes, that's correct. $\endgroup$ Commented May 22, 2017 at 7:27
$\begingroup$
$\endgroup$
4
From a geometric perspective 2 vectors are orthogonal if they are perpendicular to one another.
answered Jan 21, 2015 at 8:25
-
2$\begingroup$ This does not provide an answer to the question. To critique or request clarification from an author, leave a comment below their post. $\endgroup$– Xi'anCommented Jan 21, 2015 at 8:52
-
$\begingroup$ Sure it does. I've provided another way for the OP to understand what it means when two random variables are orthogonal. $\endgroup$ Commented Jan 21, 2015 at 10:10
-
2$\begingroup$ But what does it mean to say that two distributions are perpendicular? How can I view two distributions as vectors? Do I need to consider Hilbert space to do that? Actually my knowledge about Hilbert space is very limited. So can you please explain? $\endgroup$– arnabCommented Jan 21, 2015 at 10:55
-
2$\begingroup$ arnab Although this answer is not wrong, it's not terribly informative either. As you point out, one has to show the sense in which random variables (not distributions!) can be thought of as "vectors" and also to show how independence is related to some kind of inner product. Another answer has already gone considerably far down that road, so if you want further clarification I suggest you ask it there. $\endgroup$– whuber ♦Commented Jan 21, 2015 at 15:58