# What does orthogonal mean in the context of statistics?

In other contexts, orthogonal means "at right angles" or "perpendicular".

What does orthogonal mean in a statistical context?

Thanks for any clarifications.

-
Thanks for the question. I have asked a more general one: what is so common among all cases of orthogonality. I was also interested to know how does statistical independence satisfy this property? physics.stackexchange.com/questions/67506 –  Val Jun 30 at 14:52
I am surprised that none of the answers here mention that usually it is meant in the mathematical "linear algebra" sense of the word. For example, when we speak of an "orthogonal set of variables" usually it is meant that $X^{T}X=I$ for the matrix with the set of variables $X$. "orthonormal" is used as well. –  probabilityislogic Sep 13 at 7:25
@probability "Orthogonal" has meaning for a vector space with a quadratic form $Q$: two vectors $v$ and $w$ are orthogonal if and only if $Q(v,w)=0$. "Orthonormal" means in addition that $Q(v,v)=1=Q(w,w)$. Thus "orthogonal" and "orthonormal" are not synonymous, nor are they restricted to finite matrices. (E.g., $v$ and $w$ may be elements of a Hilbert space, such as the space of $L^2$ complex-valued functions on $\mathbb{R}^3$ used in classical quantum mechanics.) –  whuber Sep 13 at 15:02

It means they are independent to each other. Independent variables are often considered to be at right angles to each other. For example on the X-Y plane the X and Y axis are said to be orthogonal because if a given point's x value changes, its y value remains the same, and vice versa (i.e. they are independent). See also http://en.wikipedia.org/wiki/Independence_(probability_theory) and http://en.wikipedia.org/wiki/Orthogonality

-
Because the distinction between correlation and lack of dependence is important, equating orthogonality with independence is not a good thing to do. –  whuber Jun 20 '11 at 20:26

@Mien already provided an answer, and, as pointed out by @whuber, orthogonal means uncorrelated. However, I really wish people would provide some references. You might consider the following links helpful since they explain the concept of correlation from a geometric perspective.

-

I can't make a comment because I don't have enough points, so I'm forced to speak my mind as an answer, please forgive me. From the little I know, I disagree with the selected answer by @crazyjoe because orthogonality is defined as

$$E[XY^{\star}] = 0$$

So:

If $Y=X^2$ with symmetric pdf they they are dependent yet orthogonal.

If $Y=X^2$ but pdf zero for negative values, then they dependent but not orthogonal.

Therefore, orthogonality does not imply independence.

-

If X and Y are independent then they are Orthogonal. But the converse is not true as pointed out by the clever example of user497804. For the exact definitions refer to

Orthogonal : Complex-valued random variables $C_1$ and $C_2$ are called orthogonal if they satisfy ${\rm cov}(C_1,C_2)=0$

(Pg 376, Probability and Random Processes by Geoffrey Grimmett and David Stirzaker)

Independent: The random variables $X$ and $Y$ are independent if and only if $F(x,y) = F_X(x)F_Y(y)$ for all $x,y \in \mathbb{R}$

which, for continuous random variables, is equivalent to requiring that $f(x,y) = f_X(x)f_Y(y)$

(Page 99, Probability and Random Processes by Geoffrey Grimmett and David Stirzaker)

-

It's most likely they mean 'unrelated' if they say 'orthogonal'; if two factors are orthogonal (e.g. in factor analysis), they are unrelated, their correlation is zero.

-
The correlation coefficient is (or is naturally interpretable as) the cosine of an angle. When it is zero, what do you think the angle is? :-) Uncorrelated does not mean unrelated! –  whuber Jun 20 '11 at 14:14
I'm not saying you're wrong, but could you give me an example of something that's uncorrelated and related; or vice versa? I'm not sure I understand the difference. –  Mien Jun 20 '11 at 14:43
And yes, I know that that angle would be 90°. A right angle is orthogonal. –  Mien Jun 20 '11 at 14:57
Let $X$ be a random variable taking values in $\{-1,0,1\}$ with equal probability and let $Y=X^2$. The correlation between $X$ and $Y$ is $\rho_{X,Y}=0$, but clearly they are related: $Y$ is a function of $X$. –  Max Jun 20 '11 at 19:34
Ah yes, thank you. But the opposite isn't possible, is it (if there isn't a third variable or something similar)? –  Mien Jun 20 '11 at 20:23