If $\beta_1, \beta_2$ and $\beta_3$ are three angles such that $\beta_3=\beta_1+\beta_2$ (for example, as in the outer angle of a triangle) and we have available measurements $Y_1, Y_2, Y_3$ of $\beta_1, \beta_2, \beta_3$, respectively. Due to measurement error, $Y_1 + Y_2$ might not be equal to $Y_3$. If it is assumed that $Y_i \sim N(\beta_i, \sigma^2)$, $i=1,2,3$ independently, how to derive the least square estimates $\hat{\beta_1}, \hat{\beta_2}, \hat{\beta_3}$ and unbiased estimate $\hat{\sigma^2}$?


1 Answer 1


Your least squares loss function is $$ \ell(\beta_1,\beta_2) = (Y_1 - \beta_1)^2 + (Y_2 - \beta_2)^2 + (Y_3 - \beta_1 - \beta_2)^2 $$ Now, it's not too difficult to take derivative wrt to $\beta_1$ and $\beta_2$, set them both to zero, and solve the system of 2 equations and 2 unknowns. After doing this you will obtain $$ \hat\beta_1 = \dfrac{2Y_1 - Y_2 + Y_3}{3} \quad\text{and}\quad \hat\beta_2 = \dfrac{2Y_2 - Y_1 + Y_3}{3}. $$ It's also easy to see that these are unbiased, since $E(\hat\beta_1) = \beta_1$ and $E(\hat\beta_2) = \beta_2$. Now, the estimate of $\sigma^2$ will be some multiple of $\ell(\hat\beta_1,\hat\beta_2)$, which after rearranging turns out to be $$ \ell(\hat\beta_1,\hat\beta_2) = \left( \dfrac{(Y_1-\beta_1) + (Y_2-\beta_2) - (Y_3 - \beta_1 - \beta_2)}{\sqrt{3}} \right)^2 $$ This means that $$ \dfrac{1}{\sigma^2}\ell(\hat\beta_1,\hat\beta_2)\sim\chi^2_1. $$ Since the mean of a chi-square with 1 degrees of freedom is 1, we have an unbiased estimator of $\sigma^2$ as $$ \hat\sigma^2 = \ell(\hat\beta_1,\hat\beta_2), $$ which van also be written as $$ \hat\sigma^2 = \left( \dfrac{Y_1 + Y_2- Y_3 }{\sqrt{3}} \right)^2. $$

  • 3
    $\begingroup$ Perhaps writing $\hat\beta_1 = Y_1-\dfrac{ Y_1 + Y_2-Y_3}{3}$ and $\hat\beta_2 = Y_2-\dfrac{ Y_1 + Y_2-Y_3}{3}$ and $\hat\beta_3 = Y_3+\dfrac{ Y_1 + Y_2-Y_3}{3} = \hat\beta_1+\hat\beta_2$ might make it clearer what is going on. You take the difference and spread it equally between the three measurements (they have identically distributed errors) to get angles which add up $\endgroup$
    – Henry
    Oct 10, 2021 at 14:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.