Confusion about a least squares estimation problem

Question

If $\beta_1, \beta_2$ and $\beta_3$ are three angles such that $\beta_3=\beta_1+\beta_2$ (for example, as in the outer angle of a triangle) and we have available measurements $Y_1, Y_2, Y_3$ of $\beta_1, \beta_2, \beta_3$, respectively. Due to measurement error, $Y_1 + Y_2$ might not be equal to $Y_3$. If it is assumed that $Y_i \sim N(\beta_i, \sigma^2)$, $i=1,2,3$ independently, how to derive the least square estimates $\hat{\beta_1}, \hat{\beta_2}, \hat{\beta_3}$ and unbiased estimate $\hat{\sigma^2}$?

Answer 1

Your least squares loss function is $$ \ell(\beta_1,\beta_2) = (Y_1 - \beta_1)^2 + (Y_2 - \beta_2)^2 + (Y_3 - \beta_1 - \beta_2)^2 $$ Now, it's not too difficult to take derivative wrt to $\beta_1$ and $\beta_2$, set them both to zero, and solve the system of 2 equations and 2 unknowns. After doing this you will obtain $$ \hat\beta_1 = \dfrac{2Y_1 - Y_2 + Y_3}{3} \quad\text{and}\quad \hat\beta_2 = \dfrac{2Y_2 - Y_1 + Y_3}{3}. $$ It's also easy to see that these are unbiased, since $E(\hat\beta_1) = \beta_1$ and $E(\hat\beta_2) = \beta_2$. Now, the estimate of $\sigma^2$ will be some multiple of $\ell(\hat\beta_1,\hat\beta_2)$, which after rearranging turns out to be $$ \ell(\hat\beta_1,\hat\beta_2) = \left( \dfrac{(Y_1-\beta_1) + (Y_2-\beta_2) - (Y_3 - \beta_1 - \beta_2)}{\sqrt{3}} \right)^2 $$ This means that $$ \dfrac{1}{\sigma^2}\ell(\hat\beta_1,\hat\beta_2)\sim\chi^2_1. $$ Since the mean of a chi-square with 1 degrees of freedom is 1, we have an unbiased estimator of $\sigma^2$ as $$ \hat\sigma^2 = \ell(\hat\beta_1,\hat\beta_2), $$ which van also be written as $$ \hat\sigma^2 = \left( \dfrac{Y_1 + Y_2- Y_3 }{\sqrt{3}} \right)^2. $$