0
$\begingroup$

I recently saw a question on the scikit-learn mailing list that I had wondered about. This is the formula to minimize the residual sum of squares.

http://scikit-learn.org/stable/modules/linear_model.html

The formula for the minimization is:

$$ \min _w \| Xw - y \|_2^2 $$

I think the formula says that we retain the minimum set of coefficients ($w$) that we found from the smallest squared difference of predicted responses minus the observed values, which is $\|Xw -y\|^2$

What does the subscript 2 in the formula refer to?

Improve this question
$\endgroup$
3
  • 4
    $\begingroup$ $\| x \|_2$ is the $L^2$ norm of the vector $x$ so $\| x \|_2^2$ is its square. This is just a symbolic way of saying find the vector $w$ that minimizes the sum of the squared differences between the elements $Xw$ and $y$. $\endgroup$ – dsaxton Mar 7 '16 at 19:06
  • $\begingroup$ dsaxton's comment above answered the question. Thanks! $\endgroup$ – user2434291 Mar 8 '16 at 5:00
  • $\begingroup$ @dsaxton Since you seem to have answered the question to the satsifaction of the OP, could you consider reframing that as an answer? $\endgroup$ – Glen_b Mar 8 '16 at 7:26
1
$\begingroup$

$\| x \|_2$ is the $L^2$ norm of the vector $x$ and is equal to $\sqrt{\sum_{i=1}^{p} x_i^2}$ where $x_i$ is the $i^\text{th}$ element of $x$, and so $\| x \|_2^2$ is the sum of squares $\sum_{i=1}^{p} x_i^2$.

$\min_w \| Xw - y \|_2^2$ is then just a symbolic way of saying "the vector $w$ which minimizes the sum of squared differences between the elements of $Xw$ and $y$."

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.