# Do predicted values differ if variables included stepwise or simultaneously?

You make a linear model #1 of the following form to predict adult human height:

$$y = \beta_{0.1} + \beta_{1.1}\text{Height}_\text{dad} + \beta_{2.1}\text{Height}_\text{mom}$$ You calculate fitted values $\hat{y_1}$.

Now you add $\beta_3x_3$ to make model #2 from the same data.

$$y = \beta_{0.2} + \beta_{1.2}\text{Height}_\text{dad} + \beta_{2.2}\text{Height}_\text{mom} + \beta_{3.2}\text{Age}$$ You calculate fitted values $\hat{y_2}$.

Now you make model #3 from the same data, but model #3 uses the fitted values from model #1 instead of $Height_{dad}$ and $Height_{mom}$.

$$y = \beta_{0.3} + \beta_{4.3}\hat{y_1} + \beta_{3.3}\text{Age}$$

You calculate fitted values $\hat{y_3}$.

• In terms of prediction, what is the conceptual difference between the predicted values from models #2 and #3?
• When would model #3 be appropriate, if ever?
• Short answer is no (to the question as written in the body). Mar 9 '15 at 16:34
• @Affine: question has been improved
– jtd
Mar 10 '15 at 22:15

Everything depends on which parameters are relevant in explaining the dependent variable.

Case 1: Age is not relevant. $\beta_3$ = 0

In this case it shouldn't matter if you do it stepwise or simultaneously.

Case 2: Age is relevant $\beta_3\neq$ 0, but Age is orthogonal to other regressors in the model.

In this case it shouldn't also matter if you do it stepwise or simultanously.

In other cases results should be different.

I am also assuming that the estimates are consistent, i.e. this is not an errors-in-variables model (regressors are not measured with an error), or if there are other relevant regressors that are missing from the model, these are orthogonal to those that are included in the model.

Follow the link for an in depth treatment Endogeneity.

• Fascinating, "But the OLS estimator $\hat{\beta_1}$ is a biased and inconsistent estimator of $\beta_1$ in the true model, except in the special case where all of the [\$\beta \ne 0 ??] omitted variables are orthogonal to all of the included variables."
– jtd
Mar 12 '15 at 13:11
• @jtd Does this answer your question, or is there some part that is not covered? Mar 12 '15 at 13:28
• Apologies. Your answer claims there is no conceptual difference as long as all (significant) omitted variables are orthogonal to all included variables but I still haven't understood the conceptual difference if that assumption fails (which may be quite common). Is this a standard assumption (noting that linearly independent, uncorrelated, and orthogonal are all different) or a strong assumption? (psych.umn.edu/faculty/waller/classes/FA2010/Readings/…)
– jtd
Mar 12 '15 at 15:00
• @jtd I don't quite understand what you are asking. In general when we are building a model there is always an implicit assumption that we are including what's necessary. But since we cannot know if we had left out something important in advance, we can at least check whatever we left out doesn't cause wrong (biased is more correct term) coefficients for those we included. For this we can check whether the residuals of the regression is correlated to any of the included variables. Mar 12 '15 at 15:29
• You suggest that if residuals follow classical assumptions, then model meets orthogonality requirement, and thus models #2 and #3 are equivalent in all cases?
– jtd
Mar 12 '15 at 15:47