What does orthogonal mean in the context of statistics?

Question

In other contexts, orthogonal means "at right angles" or "perpendicular".

What does orthogonal mean in a statistical context?

Thanks for any clarifications.

Answer 1

It means they are independent to each other. Independent variables are often considered to be at right angles to each other. For example on the X-Y plane the X and Y axis are said to be orthogonal because if a given point's x value changes, its y value remains the same, and vice versa (i.e. they are independent). See also http://en.wikipedia.org/wiki/Independence_(probability_theory) and http://en.wikipedia.org/wiki/Orthogonality

Answer 2

@Mien already provided an answer, and, as pointed out by @whuber, orthogonal means uncorrelated. However, I really wish people would provide some references. You might consider the following links helpful since they explain the concept of correlation from a geometric perspective.

• The Geometry of Vectors (see p. 7)
• Linearly Independent, Orthogonal, and Uncorrelated Variables
• Graphical representation of two-dimensional correlation in vector space (may not be free to you)

Answer 3

I can't make a comment because I don't have enough points, so I'm forced to speak my mind as an answer, please forgive me. From the little I know, I disagree with the selected answer by @crazyjoe because orthogonality is defined as

$$E[XY^{\star}] = 0$$

So:

If $Y=X^2$ with symmetric pdf they they are dependent yet orthogonal.

If $Y=X^2$ but pdf zero for negative values, then they dependent but not orthogonal.

Therefore, orthogonality does not imply independence.

Answer 4

If X and Y are independent then they are Orthogonal. But the converse is not true as pointed out by the clever example of user497804. For the exact definitions refer to

Orthogonal : Complex-valued random variables $C_1$ and $C_2$ are called orthogonal if they satisfy ${\rm cov}(C_1,C_2)=0$

(Pg 376, Probability and Random Processes by Geoffrey Grimmett and David Stirzaker)

Independent: The random variables $X$ and $Y$ are independent if and only if $F(x,y) = F_X(x)F_Y(y)$ for all $x,y \in \mathbb{R}$

which, for continuous random variables, is equivalent to requiring that $f(x,y) = f_X(x)f_Y(y)$

(Page 99, Probability and Random Processes by Geoffrey Grimmett and David Stirzaker)

Answer 5

It's most likely they mean 'unrelated' if they say 'orthogonal'; if two factors are orthogonal (e.g. in factor analysis), they are unrelated, their correlation is zero.