4
$\begingroup$

I want to understand the relationship between X and Y, with a control variable Z. What are the differences (in assumption, interpretation of $\beta$, etc) between (1) the regression model of $Y = \beta X + Z$, and (2) the weighted regression model $Y = \beta X$ weighted by Z?

$\endgroup$

1 Answer 1

3
$\begingroup$

Ok, this question could be a bit clearer, because people talk about weighted regression in multiple senses. I'm going to guess you mean probability weights, in some sort of inverse-probability-of-treatment sense.

If you are talking linear regression, there's not a lot of difference. Write $\beta_R$ for the $\beta$ in $$E[Y]=\beta_R X$$ where $X$ is independent of $Z$ (ie randomised). Fitting $$E[Y]=\beta X$$ with weights $1/\pi_i$ where $\pi_i=P(X=x|Z=z)$ and fitting $$E[Y]=\beta X+\gamma Z$$ will both estimate $\beta_R$ if there are no unmeasured confounders and $X$ does not affect $Z$. Under much weaker assumptions, the two will estimate the same parameter, though it will no longer have a causal interpretation.

However, the variances of the two estimators will be different. If $Z$ is strongly correlated with $Y$ conditional on $X$, then adjusting for $Z$ in the model will reduce the residual variance and lead to more precise estimates of $\beta$ than you get with weighting. (It's possible to adjust the weights and recover this extra information in the weighted estimator using a more complicated technique called calibration of weights)

Everything is a bit different if you have logistic regression (or anything other than linear or log-linear regression), because then the true values of the adjusted and weighted $\beta$ are different, a phenomenon called non-collapsibility. In logistic regression, the adjusted one is further from zero. (They will be in the same direction.)

It's still true that you gain something by adjusting for $Z$, but what you gain is a bit harder to specify precisely, because the target parameters are different and it's no longer meaningful to just compare their standard errors. One straightforward aspect of what you gain is that the power for testing $\beta=0$ is greater when you adjust than when you weight (but nowhere near as impressively as is possible in linear regression)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.