5
$\begingroup$

I know what it means to say two variables are independent, but can't understand what does it mean to say two variables are orthogonal.Can anyone help me?

$\endgroup$
15
$\begingroup$

In order to speak about orthogonality you need to define an inner product first.

If we consider random variables with finite second moment, covariance can be shown to be an inner product. In this case two random variables are orthogonal if and only if they are uncorrelated:

$$ 0 = \text{cov}(X, Y) = \mathbb{E}[X Y] - \mathbb{E}[X] \mathbb{E}[Y] $$

Note that zero covariance does not imply independence (in general). For details, see Covariance and independence?

Also, one could have defined inner product differently, which would lead to another notion of orthogonality. Yet, the one I described, seems more common to me.

$\endgroup$
4
  • $\begingroup$ Sorry, I don't understand. The dot product is sum(X_i*Y_i), but how does this relates to E[XY]-E[X]E[Y]? $\endgroup$
    – Mario GS
    May 3 '17 at 19:10
  • $\begingroup$ @MarioGS, sum(X_i Y_i) is a special case of dot product for Euclidean vector spaces. However, there are other kinds of vector spaces (space of random variables with finite 2nd moment is not Euclidean), often equipped with a notion of a dot product $\endgroup$ May 4 '17 at 22:25
  • $\begingroup$ In that case, if X and Y are random vectors, their dot product is not defined by the euclidian definition of dot product such as X.Y=Sum(Xi*Yi), but actually as X.Y=E[X,Y]-E[X]E[Y], is that right? $\endgroup$
    – Mario GS
    May 10 '17 at 10:56
  • $\begingroup$ @MarioGS, yes, that's correct. $\endgroup$ May 22 '17 at 7:27
-2
$\begingroup$

From a geometric perspective 2 vectors are orthogonal if they are perpendicular to one another.

$\endgroup$
4
  • 2
    $\begingroup$ This does not provide an answer to the question. To critique or request clarification from an author, leave a comment below their post. $\endgroup$
    – Xi'an
    Jan 21 '15 at 8:52
  • $\begingroup$ Sure it does. I've provided another way for the OP to understand what it means when two random variables are orthogonal. $\endgroup$ Jan 21 '15 at 10:10
  • 2
    $\begingroup$ But what does it mean to say that two distributions are perpendicular? How can I view two distributions as vectors? Do I need to consider Hilbert space to do that? Actually my knowledge about Hilbert space is very limited. So can you please explain? $\endgroup$
    – arnab
    Jan 21 '15 at 10:55
  • 2
    $\begingroup$ arnab Although this answer is not wrong, it's not terribly informative either. As you point out, one has to show the sense in which random variables (not distributions!) can be thought of as "vectors" and also to show how independence is related to some kind of inner product. Another answer has already gone considerably far down that road, so if you want further clarification I suggest you ask it there. $\endgroup$
    – whuber
    Jan 21 '15 at 15:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.