# Confusion about a least squares estimation problem

If $$\beta_1, \beta_2$$ and $$\beta_3$$ are three angles such that $$\beta_3=\beta_1+\beta_2$$ (for example, as in the outer angle of a triangle) and we have available measurements $$Y_1, Y_2, Y_3$$ of $$\beta_1, \beta_2, \beta_3$$, respectively. Due to measurement error, $$Y_1 + Y_2$$ might not be equal to $$Y_3$$. If it is assumed that $$Y_i \sim N(\beta_i, \sigma^2)$$, $$i=1,2,3$$ independently, how to derive the least square estimates $$\hat{\beta_1}, \hat{\beta_2}, \hat{\beta_3}$$ and unbiased estimate $$\hat{\sigma^2}$$?

Your least squares loss function is $$\ell(\beta_1,\beta_2) = (Y_1 - \beta_1)^2 + (Y_2 - \beta_2)^2 + (Y_3 - \beta_1 - \beta_2)^2$$ Now, it's not too difficult to take derivative wrt to $$\beta_1$$ and $$\beta_2$$, set them both to zero, and solve the system of 2 equations and 2 unknowns. After doing this you will obtain $$\hat\beta_1 = \dfrac{2Y_1 - Y_2 + Y_3}{3} \quad\text{and}\quad \hat\beta_2 = \dfrac{2Y_2 - Y_1 + Y_3}{3}.$$ It's also easy to see that these are unbiased, since $$E(\hat\beta_1) = \beta_1$$ and $$E(\hat\beta_2) = \beta_2$$. Now, the estimate of $$\sigma^2$$ will be some multiple of $$\ell(\hat\beta_1,\hat\beta_2)$$, which after rearranging turns out to be $$\ell(\hat\beta_1,\hat\beta_2) = \left( \dfrac{(Y_1-\beta_1) + (Y_2-\beta_2) - (Y_3 - \beta_1 - \beta_2)}{\sqrt{3}} \right)^2$$ This means that $$\dfrac{1}{\sigma^2}\ell(\hat\beta_1,\hat\beta_2)\sim\chi^2_1.$$ Since the mean of a chi-square with 1 degrees of freedom is 1, we have an unbiased estimator of $$\sigma^2$$ as $$\hat\sigma^2 = \ell(\hat\beta_1,\hat\beta_2),$$ which van also be written as $$\hat\sigma^2 = \left( \dfrac{Y_1 + Y_2- Y_3 }{\sqrt{3}} \right)^2.$$
• Perhaps writing $\hat\beta_1 = Y_1-\dfrac{ Y_1 + Y_2-Y_3}{3}$ and $\hat\beta_2 = Y_2-\dfrac{ Y_1 + Y_2-Y_3}{3}$ and $\hat\beta_3 = Y_3+\dfrac{ Y_1 + Y_2-Y_3}{3} = \hat\beta_1+\hat\beta_2$ might make it clearer what is going on. You take the difference and spread it equally between the three measurements (they have identically distributed errors) to get angles which add up Commented Oct 10, 2021 at 14:05