Consider the regression model: $$ y_i=bx_i+e_i,1\leq i\leq n.$$

where $x_i$'s are fixed non-zero real numbers and $e_i$'s are independent random variables with mean zero and equal variance.

$(a)$Consider estimator of the form $T=\sum_{i=1}^{n}a_iy_i$(where $a_i$'s are non random real numbers)that are unbiased for $b$.Show that the least square estimator of $b$ has the minimum variance in this class of estimators.

Minimizing $\sum e_i^{2}=\sum(y_i-bx_i)^2$ $w.r.t$ $b$,we get the least square estimate of $b$ as, $$\hat b=\frac{\sum y_ix_i}{\sum x_i^{2}}$$ Now,$var(\hat b)=\frac{\sigma^{2}}{\sum x_i^{2}}$,where $\sigma^2=var(y),$I assume.

Now,I tried to minimize $Var(T)=\sum a_i^2\sigma^{2}$ subject to $\sum a_i=1$ and I am getting $a_i=\frac{1}{n}$,i.e.,$T=\bar y$ as the $BLUE$ of $b$.I don't know where I am making the mistake.May be,my approach is not right.

  • $\begingroup$ $T=\hat b$ in your problem, not $T=\bar y$. $\endgroup$ Jun 6, 2016 at 14:22
  • $\begingroup$ @ChristophHanck: We know that least square estimate is the BLUE.Could you please tell me how can we find the BLUE from the class $\sum a_iy_i$ and show that it is equal to $\hat b?$ $\endgroup$
    – priyanka
    Jun 6, 2016 at 14:35
  • 1
    $\begingroup$ Have you heard of the Gauß-Markov theorem? $\endgroup$ Jun 6, 2016 at 15:13

1 Answer 1


Your minimum variance problem is: $$ \begin{equation} \begin{array}{*2{>{\displaystyle}r}} \mbox{minimize (over $a_i$)} & \mathrm{Var}\left( \sum_i a_i y_i \; \middle| \; \{x_i\}\right) \\ \mbox{subject to} & \mathrm{E}\left[\sum_i a_iy_i \; \middle|\;\{x_i\}\right] = b \end{array} \end{equation}$$ Remember you can always substitute $y_i = b x_i + \epsilon_i$ $$ \begin{equation} \begin{array}{*2{>{\displaystyle}r}} \mbox{minimize (over $a_i$)} & \mathrm{Var}\left( \sum_i a_i (bx_i + \epsilon_i) \; \middle| \; \{x_i\}\right) \\ \mbox{subject to} & \mathrm{E}\left[\sum_i a_i(bx_i + \epsilon_i) \; \middle|\;\{x_i\}\right] = b \end{array} \end{equation}$$

Simplifying, you can show this is an equivalent problem to: $$ \begin{equation} \begin{array}{*2{>{\displaystyle}r}} \mbox{minimize (over $a_i$)} & \sum_i a_i^2 \\ \mbox{subject to} & \sum_i a_i x_i = 1 \end{array} \end{equation}$$

Which will have a nice solution! You'll see that the solution $\mathbf{a}^*$ to the above optimization problem will make your estimator $T$ equal to $\hat{b}$.


This site is temporarily in read-only mode and not accepting new answers.

Not the answer you're looking for? Browse other questions tagged .