Some time ago I read this great short paper on the relationship between 'correlation', ‘independence’ and ‘orthogonality’. In short, two vectors can be uncorrelated, they can be orthogonal, they can be both, or they can be neither.
I have noticed in differing regression textbooks the authors state different requirements for ensuring that coefficient values are not affected by the inclusion of additional independent variables. For example, in the textbook Introduction to Econometrics by Stock & Watson the authors state that if regressor variables are uncorrelated with one another then their coefficient estimates will be invariant with respect to the inclusion of additional independent variables. In the book Introduction to Linear Regression Analysis by Montgomery the author says that regressor variables must be orthogonal to one another in order to ensure that their coefficient estimates are not affected by one another. The professor who taught the class that used the second book says that when the book described orthogonality as being necessary, it was referring to the centered versions of the regressor variables. If that is true then it is in fact uncorrelated variables that have invariant coefficient estimates - at least with respect to the inclusion of additional variables, as the first book suggests.
When studying Design of Experiments one learns that the design matrices are specifically created in such a way that the coefficient estimates associated with a regressor variable are invariant with respect to the other variables used in the analysis. Interestingly, though, these design matrices don't just ensure that the IV's are uncorrelated (which is all that is necessary to achieve the result), they also ensure orthogonality.
My question is: is there a greater purpose for ensuring orthogonality in addition to 'uncorrelated-ness'? And/or is it just too difficult to ensure 'uncorrelated-ness' without also ensuring orthogonality?