# Why does order of entry of predictor variables matter in Regression?

In my textbook, several methods such as Hierarchial, Forced Entry etc are discussed. It says:

"When predictors are all completely uncorrelated, the order of variable entry has very little effect on the parameters calculated; however, we rarely have uncorrelated predictors and so the method of predictor selection is crucial."

For Hierarchial specifically, it says:

"In hierarchical regression, predictors are selected based on past work and the experimenter decides in which order to enter the predictors into the model....... New predictors can be entered either all in one go, in a stepwise manner, or hierarchically (such that the new predictor suspected to be the most important is entered first)."

A regression equation is just an equation and the variables added as "+ beta * variable". How can the order even matter? Does it have anything to do with semi-partial correlation?

Proof: For two centered predictor samples $$x_1$$ and $$x_2$$ satisfying $$\mu_{x_1} = \mu_{x_2}=0$$, the sample correlation coefficient will be: $$COR(x_1,x_2) = (x_1 - \mu_{x_1})^T(x_2 - \mu_{x_2})/(\sigma_{x_1}\sigma_{x_2}) = x_1^Tx_2/(\sigma_{x_1}\sigma_{x_2})$$ So when $$COR(x_1,x_2)=0$$, that means $$x_1^Tx_2=0$$, thus $$x_1$$ and $$x_2$$ are orthogonal to each other; On the other hand, when $$x_1$$ and $$x_2$$ are orthogonal, $$x_1^Tx_2=0$$, so that $$COR(x_1,x_2)=0$$.
And when samples of the predictors are orthogonal to eacher, their coefficients in the multivartiate regression will be the same as the ones in the univariate regression, thus the order of the predictor doesn't matter. Say if $$x_1 \perp x_2$$, then when fitting $$y = \beta_1^{(1)} x_1 + \beta_2^{(1)} x_2 \\ y = \beta_1^{(2)} x_1 \\ y= \beta_2^{(2)} x_2$$ you will get $$\beta_1^{(1)} = \beta_1^{(2)}$$ and $$\beta_2^{(1)}=\beta_2^{(2)}$$.
This statement is no longer true when $$x_1 \not\perp x_2$$. For example when doing regression by successive orthogonalization, the predictors entered earlier will have an impact on the coefficient of the predictors entered later, if they are not orghogonal to each other.