# Minimizing residual sum of squares formula

I recently saw a question on the scikit-learn mailing list that I had wondered about. This is the formula to minimize the residual sum of squares.

http://scikit-learn.org/stable/modules/linear_model.html

The formula for the minimization is:

$$\min _w \| Xw - y \|_2^2$$

I think the formula says that we retain the minimum set of coefficients ($w$) that we found from the smallest squared difference of predicted responses minus the observed values, which is $\|Xw -y\|^2$

What does the subscript 2 in the formula refer to?

• $\| x \|_2$ is the $L^2$ norm of the vector $x$ so $\| x \|_2^2$ is its square. This is just a symbolic way of saying find the vector $w$ that minimizes the sum of the squared differences between the elements $Xw$ and $y$. Mar 7, 2016 at 19:06
• dsaxton's comment above answered the question. Thanks! Mar 8, 2016 at 5:00
• @dsaxton Since you seem to have answered the question to the satsifaction of the OP, could you consider reframing that as an answer? Mar 8, 2016 at 7:26

$\| x \|_2$ is the $L^2$ norm of the vector $x$ and is equal to $\sqrt{\sum_{i=1}^{p} x_i^2}$ where $x_i$ is the $i^\text{th}$ element of $x$, and so $\| x \|_2^2$ is the sum of squares $\sum_{i=1}^{p} x_i^2$.
$\min_w \| Xw - y \|_2^2$ is then just a symbolic way of saying "the vector $w$ which minimizes the sum of squared differences between the elements of $Xw$ and $y$."