# Confusion regarding least squares method

I am having some confusion regarding least squares method. Actually, least squares method is for minimizing the square of the $L_2$ norm of $Ax-b$ as given in this video lecture. However I am confused if we are given a set of points like $(x_1,y_1),(x_2,y_2)...(x_4,y_4)$. Then our model is something like

$$y=ax+b$$ where we have to find the parameters and b. So we need to minimize the square of the $L_2$ norm of

$$(y-ax-b)$$

isn't it?

## 1 Answer

Minimizing $(Ax-b)^2$ is wrong; minimizing $(y-(Ax+b))^2$ is right.

• It's in this video lecture stanford.edu/class/ee364a/videos/video01.html. This is really popular. I think I might have misinterpreted something. I just need some clarifications – user31820 Jun 19 '12 at 16:59
• Oh, that's just notation. They're doing plain convex optimization, not linear regression, so they're defining "b" to be (y-b) in the linear regression case. In the problem they're solving, there is no "y". Is that clear? They're just solving for x; A and b are assumed known. (edit: last statement was wrong) – Johann Hibschman Jun 19 '12 at 17:02
• Can you give me a good working example? – user31820 Jun 19 '12 at 17:11
• They're not solving linear regression. They're doing a math problem. There's no need for them to mention "y" in this context at all. The only problem they have is to minimize Ax-b. Linear regression just happens to be a problem you can reduce to that form. "A" becomes a matrix of [[1 x1] [1 x2] [1 x3] [1 x4]]; "x" is the vector of [b a]; "b" is y. – Johann Hibschman Jun 19 '12 at 17:12