I'm facing a problem where I think I might be falling into Simpson's Paradox. I remember proving Linear Regression by using partial derivatives, so I'm thinking that even though Linear Regression is a linear correlation, the effect of each parameter I'm analysing is isolated. Is that correct? Am I missing something?
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Partly, yes. Simpson's paradox was one of the earliest efforts to characterize (an extreme form of) confounding. If the general issue is: how do we deal with confounding? then linear regression is a tool but without an approach, it is useless.
Linear regression is not "proved" with partial derivatives. I think you are referring to deriving the least squares regression as the MLE of the multivariate normal likelihood. And even then we are assuming that the functional form of the response is correctly specified! That is, if there are confounders, we know them and their (conditional) relation to the response.
Linear regression is a projection. The residuals are conditionally independent (on average) of the regressors. But it is misleading to say that you "isolate" the effect of each parameter. As with Simpson's paradox, or any other confounding analysis, you have a predictor of interest X and a response or an outcome Y. A third effect W is a confounder if it is causal of X and of Y. In the linear regression of Y on X, we can call it a "crude" analysis, and with adding W an "adjusted" analysis. The interpretation of the slope and intercept is fundamentally different for the crude and adjusted models. You have not isolated the effect, you have changed what you're estimating into a more nuanced and correct outcome. The slope coefficient for the X term can only be compared in those two models when W is independent of X, which would preclude its being a confounder.
This is how adjustment can remove confounding in linear regression.
It's more complicated when you consider logistic regression or survival analysis.
