Weighted Least Squares weights not changing Jacobian matrix

Question

I currently have 4 data points and the following Jacobian matrix $A$ and cost vector $b$

$ A = \begin{bmatrix} -0.7867 & 0.0464& -0.6155 & 1.0000 \\ -0.3751 & 0.4299 & -0.8213 & 1.0000 \\ 0.0447 & 0.4895 & -0.8708 & 1.0000 \\ -0.5946 & 0.8029 & 0.0424 & 1.0000 \\ \end{bmatrix} $

$b = \begin{bmatrix} 26.3019 \\ 1.1 \\ 4.4677 \\ 4.6455 \\ \end{bmatrix}$

I am trying to use weighted least squares with the following equation to reduce the effect of the 1st data point.

$H = (A^TWA)^{-1}A^TW$ so that $y = Hb$

However my weighted Jacobian matrix $H$ seems to be virtually the same regardless of what weights I put in.

$W_1 =\begin{bmatrix} 1 & 0& 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0\\ 0 & 0 & 0 & 1 \\ \end{bmatrix}$

$H_1 =\begin{bmatrix} 0.4964 & -3.4382& 2.8939& 0.0479\\ -2.4944& 4.2279 & -2.2013& 0.4678\\ 1.2034& -3.8577& 1.6862& 0.9681\\ 2.2469& -5.2755& 3.4167& 0.6119\\ \end{bmatrix}$

$W_2 =\begin{bmatrix} .0001 & 0& 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0\\ 0 & 0 & 0 & 1 \\ \end{bmatrix}$

$H_2 =\begin{bmatrix} 0.4964 & -3.4382& 2.8939& 0.0479\\ -2.4944& 4.2279 & -2.2013& 0.4678\\ 1.2034& -3.8577& 1.6862& 0.9681\\ 2.2469& -5.2755& 3.4167& 0.6119\\ \end{bmatrix}$

Is there a reason why my weights have no effect on the Jacobian and are there requirements on the Jacobian matrix or the weights of Weighted Least Squares? For the weight matrix $W$, should it be $w^Tw$ for a 4x1 weight vector instead of a diagonal matrix?

Answer 1

This is simply because $A$ is a square and non-singular matrix. Therefore, \begin{align} H = (A^TWA)^{-1}A^TW = A^{-1}(A^TW)^{-1}A^TW = A^{-1}. \end{align} In practice, $A$ is typically slim rectangular ($n\times p$ with $n > p$) so $(A^TWA)^{-1}$ cannot be written to $A^{-1}(A^TW)^{-1}$ thus $W$ would have effect.