Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

The problem is:

Suppose we fit a model Y = XA βA + ε. However, the true model is Y = XA βA + XB βB + ε. A is kA x 1 and B is kB x 1. Show that the OLS estimates of βA will still be equal if XTAXB = 0.

My attempt at a solution:

Under OLS, the estimate of β is (XTX)-1XTY. So I was thinking for the first model, we just have X = XA, but in the second model, we have X = XA + XB. So I was trying to expand (XTX)-1XTY but subbing in the previous values for X, and then seeing if anything stood out, but I wasn't having any luck.

Any help or tips would be appreciated

share|improve this question
Are $k_A$ & $k_B$ supposed to be equal? It seems like that should be mentioned. – gung Feb 3 '14 at 3:38

In the second model $X=[X_A,X_B]$, not $X=X_A+X_B$. This is where the confusion comes in. Substitute the correct matrix into the $X^TX$ and note that the proposed condition makes the matrix diagonal like. Think what this implies. Or alternatively use Frisch–Waugh–Lovell theorem.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.