0
$\begingroup$

Frisch-Waugh's theorem states that in the setup $Y = X^T \beta $, where $\beta = [\beta_1,\beta_2]^T$, $X = [X_1, X_2]$, $\hat{\beta}_2$ obtained from the multiple regression is the same as that obtained from a two-stage specification, i.e. $\hat{\beta}_2 = (X_2^T X_2)^{-1} X_2^T(Y-X_1\hat{\beta}_1)$

This procedure should also hold for WLS:

$\hat{\beta}_2 = (X_2^T \Omega^{-1}X_2)^{-1}X_2^T \Omega^{-1}(Y-X_1\hat{\beta}_1)$

However, I find different coefficient estimates for a single multiple regression than for two separate regression.

From the equation I derived, weights should be used in both parts of the two-part regressions, but just in case, I tried adding weights to only the first or second regression as well.

A simple reproducible example in R:

# load dataset from MASS package
data(cabbages)
# Here we see exactly what the coef on Cult should be:
target <- lm(VitC ~ Date + Cult, weight=HeadWt, data=cabbages)
# We see that the coef of Cultc52 is 12.69

# Weights on both equations: 
one_f <- lm(VitC ~ Date,weight = HeadWt,data=cabbages)
(one_s <- lm(resid(one_f) ~ Cult,weight=HeadWt,data=cabbages))
# 12.42 

# Weights on the first equation only
two_f <- lm(VitC ~ Date, weight=HeadWt,data=cabbages)
(two_s <- lm(resid(two_f) ~ Cult, data=cabbages))
# 12.90... hmmm... 

# Weights on second equation only
three_f <- lm(VitC ~ Date,data=cabbages)
(three_s <- lm(resid(three_f) ~ Cult,weight=HeadWt,data=cabbages))
# 12.79
$\endgroup$

1 Answer 1

2
$\begingroup$

Your statement of Frisch-Waugh is actually incorrect; essentially you can partial $X_1$ out of $Y$, but you have to partial it out of $X_2$. you want the following:

$$ \hat{\beta}_2=(X_2M_{x1}X_2)^{-1}X_2M_{x1}Y $$

Where

$$ M_{x2}=(I-X_2(X_2X_2)^{-1}X_2X_1) $$

That does work with weights in the first and second stage.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.