What is the relationship between orthogonality and the expectation of the product of RVs

Question

Is there such thing as a statistical concept of orthogonality? Does somebody could provide a formal explanation about the relationship between orthogonality and conditional expectation of a RV? Here is the motivation for the question. In Greene (2011) Econometric Analysis, pg. 93, he writes

(1) "Assumption A3 states that the disturbances in the population are stochastically orthogonal to the independent variables in the model; that is, $E[\epsilon|\vec{x}]=0$"

A3: $E[\epsilon|\vec{x}]=E[\epsilon|x_1,x_2,...,x_n]=0$

Why $E[\epsilon|\vec{x}]=0$ is the same thing as to say that $e_i$ is stochastically orthogonal of each $x_i$, the independent variables in the model. How to random variables can be orthogonal?

Definition of orthogonality: it is defined in Linear Algebra and it requires at least two vectors. One possible way to say two vectors are orthogonal is that their dot product is zero, that is, if $x=(x_1,...,x_n)$ and $y=(y_1,...,y_n)$ then

$x\cdot y = 0$

Definition of conditional expectation:

$E[\epsilon|\vec{x}] = \int_\epsilon \epsilon f(\epsilon|\vec{x})d\epsilon$

How the two concepts are formally related?

Answer 1

Notice that orthogonality is concept that derives its self intuitions about vectors in $R^n$ were we say if dot product between two vectors is 0 then they are orthogonal. Well,in linear algebra this idea is generalized function and the dot prodcut is replaced by a general called an inner product, $\langle x,y\rangle$, (which is basically a function, letting $V$ be a vector space, $ V^2\rightarrow \mathbb{R}$ that follows certain criteria). And here we say vectors are orthogonal if their inner product of the vectors is 0.

Thus all you need to have idea of orthogonality with RVs as your vectors, is to define an inner product. Well, the usual inner product used and one that fulfills the criteria for an inner product is $$\langle X,Y \rangle= E(XY)$$ Thus RVs, X and Y are orthogonal if $E(XY)=0$

Answer 2

I think I got the answer for this question. Although orthogonality is a concept from Linear Algebra, and it means that the dot-product is zero, the term is loosely used in statistics. Whenever we have two Random Vectors $(X,\epsilon)$ with $E[X]=E[\epsilon]=0$, the correlation of X and Y is the expectation of their dot-product. If, also, $E[\epsilon|X]=0$, then it must be the case that $X\cdot\epsilon=0$. Formally, Assume $E[\epsilon|X]=0$ (the A3 of the question) and $E[\epsilon]=0$ (this last assumption is less important b/c we can always centralize a RV around its mean to make its expectation be zero). Then $$Cov(X,\epsilon|X) = E[X\cdot\epsilon|X]=X\cdot E[\epsilon|X] = 0\implies X\cdot\epsilon=0$$ If $X\cdot\epsilon\neq 0$, lets say $X\cdot\epsilon = c$ then we would have $E[X\cdot\epsilon|X] = c\neq 0$. In other words, as $E[X\cdot\epsilon|X] =0\implies X\cdot\epsilon=0$, it is called orthogonality condition.