3
$\begingroup$

When running a weighted regression, do I multiply my constant term by the weight vector?

$\endgroup$
2
  • 1
    $\begingroup$ Did you happen to come across this? $\endgroup$ Commented Jan 18, 2017 at 0:23
  • $\begingroup$ @AntoniParellada ty that was helpful. $\endgroup$
    – piRSquared
    Commented Jan 18, 2017 at 0:57

1 Answer 1

3
$\begingroup$

I see you are a Python expert, but let me illustrate the point in the link provided in R, comparing side-by-side with the matrices involved so that the answer becomes clear:

The operation to carry out is:

$$\begin{align*} \hat{\beta}_{WLS}&=\arg\min_{\beta}\sum_{i=1}^{n}\varepsilon_{i}^{2}\\ &=(\textbf{X}^{T}\textbf{W}\textbf{X})^{-1}\textbf{X}^{T}\textbf{W}\textbf{Y}. \end{align*}\tag{1}$$

So we get the data loaded as in here. It looks like this:

> galton
  Parent Progeny      SD
1   0.21  0.1726 0.01988
2   0.20  0.1707 0.01938
...

And we run the weighted linear model as:

weighted_model = lm(Progeny ~ Parent, weights= 1/SD^2, data = galton)

Now let's generate the coefficients "manually". First, the $\textbf{X}$ matrix can be extracted directly from the weighted_model object:

$$ \textbf{X}=\tiny\begin{bmatrix} \color{blue}{ \text{Intercept}} & \color{blue}{\text{Parent}} \\ 1 & 0.21 \\ 1 & 0.20 \\ 1 & 0.19 \\ 1 & 0.18 \\ 1 & 0.17 \\ 1 & 0.16 \\ 1 & 0.15 \end{bmatrix}$$

As for the matrix of weights $\textbf{W}$,

$$\textbf{W}=\tiny\begin{bmatrix} 2530.272&0&0&0&0&0&0&0\\ 0&2662.517&0&0&0&0&0&0\\ 0&0&0&2781.784&0&0&0&0\\ 0&0&0&0&2410.005&0&0&0\\ 0&0&0&0&0&3655.35&0&0\\ 0&0&0&0&0&0&3935.712&0\\ 0&0&0&0&0&0&0&3217.328 \end{bmatrix}$$

Finally, $\mathbf{Y}:$

$$\mathbf{Y}^\top=\tiny\begin{bmatrix}\color{blue}{\text{Progeny}}&0.1726 &0.1707& 0.1637& 0.1640& 0.1613 &0.1617& 0.1598\end{bmatrix}$$ ` Performing the algebraic expression in $\small (1)$ yields the intercept and the coefficients:

solve(t(X) %*% (W %*% X)) %*% (t(X) %*% (W %*% Y))
                 [,1]
(Intercept) 0.1279642
Parent      0.2048012
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.