# Estimating k in d=kv

This example was taken from Mathematical Statistics : A Unified Introduction (ISBN 9780387227696), page 58, under the section 'The Principle of Least Squares'. I think my problem has more to do with algebra but since this is a statistics book, I thought I should post here...

Anyway, I need to estimate k base on this equation:

The goal is to reach here:

I managed to follow the example until I got stuck here:

The explanation to reach to the goal from above is described as follows:

I know how to get k but I totally don't understand how the 'middle term' was eliminated. Please help. I will provide more details if needed.

There is a constraint imposed a few lines above the highlighted text which states:

$\sum_i{v_i (d_i-k v_i)} = 0$

Does the that help?

Edit

In response to your second comment:

Consider: $\sum_i{(d_i - l v_i)^2}$. This can be re-written as:

$\sum_i{((d_i - k v_i) + (k v_i - l v_i))^2}$

Expanding the square, we have:

$\sum_i{(d_i - k v_i)^2 + \sum_i(k v_i - l v_i)^2 + \sum_i{2 (d_i - k v_i) (k v_i - l v_i)}}$

Simplifying the above, we have:

$\sum_i{(d_i - k v_i)^2 + (k-l)^2 \sum_i{v_i^2} + 2 (k-l) \sum_i{v_i (d_i - k v_i)}}$

So, if we choose $k$ such that $\sum_i{v_i (d_i-k v_i)} = 0$ then it follows that:

$\sum_i{(d_i - l v_i)^2} = \sum_i{(d_i - k v_i)^2} + (k-l)^2 \sum_i{v_i^2}$

But, then it follows that:

$\sum_i{(d_i - l v_i)^2} > \sum_i{(d_i - k v_i)^2}$

as long as $k \ne l$.

Thus, what we have shown is the following: If $k$ satisfies the constraint we imposed then it must be the case that the corresponding SSE is less than the SSE for any other $l$ that we can choose. Thus, the value of $k$ that satisfies the constraint is our least squares estimate.

• I'm sorry, it does not. I don't know how 2(k-1)sum(vi(di-kvi)) can become sum(di-kvi)^2 after k was estimated. Obviously, I have failed to grasp the significance of the constraint imposed. Please explain the significance of the constraint's role in solving the equation...Thank you. – Sara Oct 21 '10 at 13:21
• "The idea is the following insight: Suppose that we choose k such that the above constraint is satisfied then it immediately follows that: ∑i(di−lvi)2=∑i(d−kvi)2+(k−l)2∑iv2i " Could you please show the intermediate steps? I really cannot see how that equation follows immediately from satisfying the constraint. Sorry for the inconvenience caused. – Sara Oct 22 '10 at 0:36