# What does orthogonal mean in the context of statistics?

In other contexts, orthogonal means "at right angles" or "perpendicular".

What does orthogonal mean in a statistical context?

Thanks for any clarifications.

-

It means they are independent to each other. Independent variables are often considered to be at right angles to each other. For example on the X-Y plane the X and Y axis are said to be orthogonal because if a given point's x value changes, its y value remains the same, and vice versa (i.e. they are independent). See also http://en.wikipedia.org/wiki/Independence_(probability_theory) and http://en.wikipedia.org/wiki/Orthogonality

-
Because the distinction between correlation and lack of dependence is important, equating orthogonality with independence is not a good thing to do. – whuber Jun 20 '11 at 20:26

@Mien already provided an answer, and, as pointed out by @whuber, orthogonal means uncorrelated. However, I really wish people would provide some references. You might consider the following links helpful since they explain the concept of correlation from a geometric perspective.

-

I can't make a comment because I don't have enough points, so I'm forced to speak my mind as an answer, please forgive me. From the little I know, I disagree with the selected answer by @crazyjoe because orthogonality is defined as

$$E[XY^{\star}] = 0$$

So:

If $Y=X^2$ with symmetric pdf they they are dependent yet orthogonal.

If $Y=X^2$ but pdf zero for negative values, then they dependent but not orthogonal.

Therefore, orthogonality does not imply independence.

-

If X and Y are independent then they are Orthogonal. But the converse is not true as pointed out by the clever example of user497804. For the exact definitions refer to

Orthogonal : Complex-valued random variables $C_1$ and $C_2$ are called orthogonal if they satisfy ${\rm cov}(C_1,C_2)=0$

(Pg 376, Probability and Random Processes by Geoffrey Grimmett and David Stirzaker)

Independent: The random variables $X$ and $Y$ are independent if and only if $F(x,y) = F_X(x)F_Y(y)$ for all $x,y \in \mathbb{R}$

which, for continuous random variables, is equivalent to requiring that $f(x,y) = f_X(x)f_Y(y)$

(Page 99, Probability and Random Processes by Geoffrey Grimmett and David Stirzaker)

-

It's most likely they mean 'unrelated' if they say 'orthogonal'; if two factors are orthogonal (e.g. in factor analysis), they are unrelated, their correlation is zero.

-
The correlation coefficient is (or is naturally interpretable as) the cosine of an angle. When it is zero, what do you think the angle is? :-) Uncorrelated does not mean unrelated! – whuber Jun 20 '11 at 14:14
I'm not saying you're wrong, but could you give me an example of something that's uncorrelated and related; or vice versa? I'm not sure I understand the difference. – Mien Jun 20 '11 at 14:43
And yes, I know that that angle would be 90°. A right angle is orthogonal. – Mien Jun 20 '11 at 14:57
Let $X$ be a random variable taking values in $\{-1,0,1\}$ with equal probability and let $Y=X^2$. The correlation between $X$ and $Y$ is $\rho_{X,Y}=0$, but clearly they are related: $Y$ is a function of $X$. – Max Jun 20 '11 at 19:34
Ah yes, thank you. But the opposite isn't possible, is it (if there isn't a third variable or something similar)? – Mien Jun 20 '11 at 20:23