# minimizer weighted linear regression

In a regression problem, with $y=X\theta+\epsilon$ and $X$ is an $n$ by $p$ matrix the ‘weighted least squares estimate is the minimizer $\theta^{*}$ of $f(\theta)=\sum_{i=1}^{n}\omega_{i}(y_i-x_i^{'}\theta)^2$ for suitable positive ‘weights’ $\omega_i$, where $x_i^{'}$ are the rows of $X$. It can be shown that if $X$ has full rank, then the minimizer $\theta^{*}=(X'WX)^{-1}X'Wy$ where $W=diag(\omega_1,\omega_2,...,\omega_n)$.

Now suppose that the observations are independent, so that the covariance matrix $\sum$ of y is diagonal, but that the diagonal elements: $\sigma_1^{2},\sigma_2^{2},...,\sigma_n^{2}$ are not all equal. We say the observations are ‘heteroscedastic’.

Question 1: Show that the covariance matrix of $\theta'$ is given by: $$COV(\theta^{*})=(X'WX)^{-1}X^{'}W\sum WX(X'WX)^{-1}$$ and that the variance of a linear combination $a\theta^{*}$ is: $$Var[a'\theta^{*}]=a'(X'WX)^{-1}X^{'}W\sum WX(X'WX)^{-1}a$$

Question 2:

Show that, for any such linear combination, $Var[a'\theta^{*}]$ is minimized by the choice of the weights: $\omega_i=1/\sigma_i^2$(i.e, when $W=\sum^{-1}$).

I solved question 1, but I have no idea about question 2. I appreciate any help. Thanks!

It's early and i'm on the bus...Maybe i'm sleeping, but i tried to solve it:

For W= S (stands for Sigma^(-1) )

a'(X'SX)^(-1) X'S S^(-1) S X(X'SX)^(-1) a =

Where S^(-1) S = Diag(1)

a'(X'SX)^(-1) X' SX(X'SX)^(-1) a =

a (X'SX)^(-1)a .

where (X'SX)^(-1) X' SX= Diag(1)

a (X'SX)^(-1)a , where (X'SX)^(-1) is homoscedastic covariance matrix of Theta vector and it is minimal cov matrix for the gauss-markov theorem.