9
$\begingroup$

This wiki page Simple linear regression has formulas to calculate $\alpha$ and $\beta$. Could anyone tell me how to derive the formulas in weighted case?

$\endgroup$
16
$\begingroup$

Think of ordinary least squares (OLS) as a "black box" to minimize

$$\sum_{i=1}^n (y_i - (\alpha 1 + \beta x_i))^2$$

for a data table whose $i^\text{th}$ row is the tuple $(1, x_i, y_i)$.

When there are weights, necessarily positive, we can write them as $w_i^2$. By definition, weighted least squares minimizes

$$\sum_{i=1}^n w_i^2(y_i - (\alpha 1 + \beta x_i))^2$$

$$=\sum_{i=1}^n (w_i y_i - (\alpha w_i + \beta w_i x_i))^2 .$$

But that's exactly what the OLS black box is minimizing when given the data table consisting of the "weighted" tuples $(w_i, w_i x_i, w_i y_i)$. So, applying the OLS formulas to these weighted tuples gives the formulas you seek.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.