Does weighted ols multiply the constant by the weights

Question

When running a weighted regression, do I multiply my constant term by the weight vector?

Answer 1

I see you are a Python expert, but let me illustrate the point in the link provided in R, comparing side-by-side with the matrices involved so that the answer becomes clear:

The operation to carry out is:

$$\begin{align*} \hat{\beta}_{WLS}&=\arg\min_{\beta}\sum_{i=1}^{n}\varepsilon_{i}^{2}\\ &=(\textbf{X}^{T}\textbf{W}\textbf{X})^{-1}\textbf{X}^{T}\textbf{W}\textbf{Y}. \end{align*}\tag{1}$$

So we get the data loaded as in here. It looks like this:

> galton
  Parent Progeny      SD
1   0.21  0.1726 0.01988
2   0.20  0.1707 0.01938
...

And we run the weighted linear model as:

weighted_model = lm(Progeny ~ Parent, weights= 1/SD^2, data = galton)

Now let's generate the coefficients "manually". First, the $\textbf{X}$ matrix can be extracted directly from the weighted_model object:

$$ \textbf{X}=\tiny\begin{bmatrix} \color{blue}{ \text{Intercept}} & \color{blue}{\text{Parent}} \\ 1 & 0.21 \\ 1 & 0.20 \\ 1 & 0.19 \\ 1 & 0.18 \\ 1 & 0.17 \\ 1 & 0.16 \\ 1 & 0.15 \end{bmatrix}$$

As for the matrix of weights $\textbf{W}$,

$$\textbf{W}=\tiny\begin{bmatrix} 2530.272&0&0&0&0&0&0&0\\ 0&2662.517&0&0&0&0&0&0\\ 0&0&0&2781.784&0&0&0&0\\ 0&0&0&0&2410.005&0&0&0\\ 0&0&0&0&0&3655.35&0&0\\ 0&0&0&0&0&0&3935.712&0\\ 0&0&0&0&0&0&0&3217.328 \end{bmatrix}$$

Finally, $\mathbf{Y}:$

$$\mathbf{Y}^\top=\tiny\begin{bmatrix}\color{blue}{\text{Progeny}}&0.1726 &0.1707& 0.1637& 0.1640& 0.1613 &0.1617& 0.1598\end{bmatrix}$$ ` Performing the algebraic expression in $\small (1)$ yields the intercept and the coefficients:

solve(t(X) %*% (W %*% X)) %*% (t(X) %*% (W %*% Y))
                 [,1]
(Intercept) 0.1279642
Parent      0.2048012