3
$\begingroup$

Is there such thing as a statistical concept of orthogonality? Does somebody could provide a formal explanation about the relationship between orthogonality and conditional expectation of a RV? Here is the motivation for the question. In Greene (2011) Econometric Analysis, pg. 93, he writes

(1) "Assumption A3 states that the disturbances in the population are stochastically orthogonal to the independent variables in the model; that is, $E[\epsilon|\vec{x}]=0$"

A3: $E[\epsilon|\vec{x}]=E[\epsilon|x_1,x_2,...,x_n]=0$

Why $E[\epsilon|\vec{x}]=0$ is the same thing as to say that $e_i$ is stochastically orthogonal of each $x_i$, the independent variables in the model. How to random variables can be orthogonal?

Definition of orthogonality: it is defined in Linear Algebra and it requires at least two vectors. One possible way to say two vectors are orthogonal is that their dot product is zero, that is, if $x=(x_1,...,x_n)$ and $y=(y_1,...,y_n)$ then

$x\cdot y = 0$

Definition of conditional expectation:

$E[\epsilon|\vec{x}] = \int_\epsilon \epsilon f(\epsilon|\vec{x})d\epsilon$

How the two concepts are formally related?

$\endgroup$
2
  • 1
    $\begingroup$ All you have to do is enlarge your concept of vector: from that (standard) definition it is immediate that random variables (defined on a common sample space) form a real vector space and that the expectation of a product is the perfect analog of the "dot product" in the finite-dimensional case. I would be curious to know what definition you would propose for orthogonality among sample spaces! $\endgroup$
    – whuber
    Dec 16 '14 at 18:35
  • 1
    $\begingroup$ This might be one of the few inaccuracies in Greene "Econometrics". It is a little inaccurate to call $E(e|x)=0$ the orthogonality condition, for it has its own name - exogeneity condition. Bit stronger than the real orthogonality condition, $E(xe)=0$. $\endgroup$
    – Khashaa
    Dec 29 '14 at 8:45
3
$\begingroup$

Notice that orthogonality is concept that derives its self intuitions about vectors in $R^n$ were we say if dot product between two vectors is 0 then they are orthogonal. Well,in linear algebra this idea is generalized function and the dot prodcut is replaced by a general called an inner product, $\langle x,y\rangle$, (which is basically a function, letting $V$ be a vector space, $ V^2\rightarrow \mathbb{R}$ that follows certain criteria). And here we say vectors are orthogonal if their inner product of the vectors is 0.

Thus all you need to have idea of orthogonality with RVs as your vectors, is to define an inner product. Well, the usual inner product used and one that fulfills the criteria for an inner product is $$\langle X,Y \rangle= E(XY)$$ Thus RVs, X and Y are orthogonal if $E(XY)=0$

$\endgroup$
1
  • 1
    $\begingroup$ There is a lot more detail to this but hopefully this gives you enough intro that you can do some of your own research $\endgroup$
    – Kamster
    Dec 29 '14 at 8:12
0
$\begingroup$

I think I got the answer for this question. Although orthogonality is a concept from Linear Algebra, and it means that the dot-product is zero, the term is loosely used in statistics. Whenever we have two Random Vectors $(X,\epsilon)$ with $E[X]=E[\epsilon]=0$, the correlation of X and Y is the expectation of their dot-product. If, also, $E[\epsilon|X]=0$, then it must be the case that $X\cdot\epsilon=0$. Formally, Assume $E[\epsilon|X]=0$ (the A3 of the question) and $E[\epsilon]=0$ (this last assumption is less important b/c we can always centralize a RV around its mean to make its expectation be zero). Then $$Cov(X,\epsilon|X) = E[X\cdot\epsilon|X]=X\cdot E[\epsilon|X] = 0\implies X\cdot\epsilon=0$$ If $X\cdot\epsilon\neq 0$, lets say $X\cdot\epsilon = c$ then we would have $E[X\cdot\epsilon|X] = c\neq 0$. In other words, as $E[X\cdot\epsilon|X] =0\implies X\cdot\epsilon=0$, it is called orthogonality condition.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.