0
$\begingroup$

Why does manually weighting a regression require the intercept term to be dropped? Consider a model

$$y=b_0 + b_1x + \epsilon, $$

a simple linear regression. In classically weighted regression only the weights are provided, say min(x)/x. In order to supply the weights manually, e.g. to a regression routine that doesn't handle weighting, a vector of weights is created, the square root of the weights is taken, then that gets multiplied to the y's, a vector of 1's, and the x's. The weighted vector of 1's is then included in the model as part of the x-matrix and the model is fit without an intercept. A parameter estimate is returned for x0 that corresponds to the intercept in a weighted regression, even though this term was not constant, and even though the intercept was omitted. Why is this the case?

$\endgroup$
0
$\begingroup$

Your "multiplication method" for handling weights only works for linear regression. Virtually every software in the world should handle weights to linear regression. If you want to trick your software into fitting this "weight-multiplied" model for you, you must suppress an intercept term. That's because when you supply a vector of weights as a covariate to the regression model, your software isn't smart enough to know this should be a "quasi-intercept term": the first column of the model matrix, so the software by default adds a constant term anyway, and this results in a different estimate of the other coefficients.

What is nominally called an intercept term and a coefficient in a model is pretty inconsequential, computationally, the intercept is just a variable of "all ones". As a matter of convenience, most software automatically adds it. You have to go through separate steps to remove it (or equivalently to add something else in it's place).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.