# Least Square Solution to Linear Regression Problem

Given two column-vectors, A = [0; 1.6818; 2.8284; 3.8337; 4.7568; 5.6234] and B = [0; 984.7; 1590.7; 2029.1; 2251.9; 2254.45], I need to find a scalar x, such that it satisfies the following condition:

$Ax = B$

Using Matrix calculation, it can be easily solved as follows:

$\boldsymbol{A}x = \boldsymbol{B}\\ \implies x = \boldsymbol{A}^{-1}\boldsymbol{B} \\ \implies x = {(\boldsymbol{A^T}\boldsymbol{A})}^{-1}\boldsymbol{A^T}\boldsymbol{B} \\ \textrm{(Taking pseudo-inverse of A)}$

Solving this gives x = 467.8599.

Which is correct, since I verified it in MATLAB using:

x = A\B
x = linsolve(A, B)

Now my question is, can I obtain the same result if I approach this problem as finding a least square solution of the regression Ax = B. That is, can I obtain x = 467.8599 by minimizing for E where,

$E = \frac{1}{2}\sum_{n=1}^{6}{{\{A_nx - B_n\}}^2}$

If yes, can you please explain how.

Also let me know if there is some mistake in my understanding. Thanks in advance.

• read the Matlab documentation, the implementations you mention perform least squares on non-square matrices – ReneBt Apr 6 '18 at 8:11
• Thanks for the pointer. Yeah, it performs QR decomposition on non-square matrices (LU Factorization if square matrices). – Koustav Apr 6 '18 at 9:24
• Yes, you just did, tow different ways. Here's another: x = pinv(A)*B – Mark L. Stone Apr 6 '18 at 11:48

You are looking for a minimum of $E$.
Therefore, just have to choose the value of $x$ such as the derivative of $E$ wrt $x$ is 0.