6
$\begingroup$

I am presenting here a very simplified problem, because I have a feeling that if I'll understand this I will be on my way to understand my original problem. I tried googeling this and also searched this site but couldn't find the answer I needed. Hopefully you'll be able to point me to the right direction.

I am solving two seemingly related problems using least square minimization given a set of random measurments $\{x_i\}$.

The first problem is finding $b$ to minimize $$\sum_i^n (x_i - b)^2 $$

The second problem is finding $c$ to minimize $$ \sum_i^n (\frac{x_i}{c} - 1)^2 $$

The solution is obviously the average of $x_i$ for $b$ $$ b = \frac{\sum_i^n x_i}{n} $$ while for $c$ it is $$ c = \frac{\sum_i^n x_i^2}{\sum_i^n x_i} $$

Naively I would expect $c$ to be the average as well. My logic: I expect $c$ to answer the question "which number, when dividing each number in the set, make the average of the resulting set closest to $1$". Obviously I'm wrong and that's not the way least-square works. Why is $c$ not the average? What does $c$ mean?

Improve this question
$\endgroup$
2
  • $\begingroup$ Well, $\sum_i^n (x_i - b)^2 = b^2\sum_i(x_i/b - 1)^2 \neq \sum_i(x_i/b - 1)^2$, so in each of these problems, you're minimizing totally different losses. I don't see a relation to these two problems. Can I ask why you're trying to find $c$? $\endgroup$
    – Taylor
    Jan 19, 2018 at 2:43
  • $\begingroup$ Basically you are beeing mislead by your intuition which seemingly ignores the ^2 $\endgroup$
    – Nikolas Rieble
    Jan 19, 2018 at 9:45

1 Answer 1

Reset to default
3
+25
$\begingroup$

In the case of b, the question being asked is "What scalar is closest to my data points (in the least squares sense)?" In the case of c, the question being asked is "How can I scale my data points to make each as close as possible to 1 (in the least squares sese)?" These objective functions are different. Imagine the case if your data had mean of 1 with extremely high variance. The optimal value of b would be 1 (up to noise), but c would not be 1 - it would necessarily be large to counter the data's high variance.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.