Consider the regression model: $$ y_i=bx_i+e_i,1\leq i\leq n.$$

where $x_i$'s are fixed non-zero real numbers and $e_i$'s are independent random variables with mean zero and equal variance.

$(a)$Consider estimator of the form $T=\sum_{i=1}^{n}a_iy_i$(where $a_i$'s are non random real numbers)that are unbiased for $b$.Show that the least square estimator of $b$ has the minimum variance in this class of estimators.

Minimizing $\sum e_i^{2}=\sum(y_i-bx_i)^2$ $w.r.t$ $b$,we get the least square estimate of $b$ as, $$\hat b=\frac{\sum y_ix_i}{\sum x_i^{2}}$$ Now,$var(\hat b)=\frac{\sigma^{2}}{\sum x_i^{2}}$,where $\sigma^2=var(y),$I assume.

Now,I tried to minimize $Var(T)=\sum a_i^2\sigma^{2}$ subject to $\sum a_i=1$ and I am getting $a_i=\frac{1}{n}$,i.e.,$T=\bar y$ as the $BLUE$ of $b$.I don't know where I am making the mistake.May be,my approach is not right.

  • $\begingroup$ $T=\hat b$ in your problem, not $T=\bar y$. $\endgroup$ Jun 6, 2016 at 14:22
  • $\begingroup$ @ChristophHanck: We know that least square estimate is the BLUE.Could you please tell me how can we find the BLUE from the class $\sum a_iy_i$ and show that it is equal to $\hat b?$ $\endgroup$
    – priyanka
    Jun 6, 2016 at 14:35
  • 1
    $\begingroup$ Have you heard of the Gauß-Markov theorem? $\endgroup$ Jun 6, 2016 at 15:13

1 Answer 1


Your minimum variance problem is: $$ \begin{equation} \begin{array}{*2{>{\displaystyle}r}} \mbox{minimize (over $a_i$)} & \mathrm{Var}\left( \sum_i a_i y_i \; \middle| \; \{x_i\}\right) \\ \mbox{subject to} & \mathrm{E}\left[\sum_i a_iy_i \; \middle|\;\{x_i\}\right] = b \end{array} \end{equation}$$ Remember you can always substitute $y_i = b x_i + \epsilon_i$ $$ \begin{equation} \begin{array}{*2{>{\displaystyle}r}} \mbox{minimize (over $a_i$)} & \mathrm{Var}\left( \sum_i a_i (bx_i + \epsilon_i) \; \middle| \; \{x_i\}\right) \\ \mbox{subject to} & \mathrm{E}\left[\sum_i a_i(bx_i + \epsilon_i) \; \middle|\;\{x_i\}\right] = b \end{array} \end{equation}$$

Simplifying, you can show this is an equivalent problem to: $$ \begin{equation} \begin{array}{*2{>{\displaystyle}r}} \mbox{minimize (over $a_i$)} & \sum_i a_i^2 \\ \mbox{subject to} & \sum_i a_i x_i = 1 \end{array} \end{equation}$$

Which will have a nice solution! You'll see that the solution $\mathbf{a}^*$ to the above optimization problem will make your estimator $T$ equal to $\hat{b}$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.