1
$\begingroup$

I have just started learning Machine Learning and one of the very first topics that I have encountered in this venture is Simple Linear Regression. From Andrew Ng's course, I have learned to perform simple linear regression using gradient descent using the loss function. But recently I was watching this YouTube video where the professor gives an equation to find the slope and the interception in 1 step.

Say if my equation is this $y=a_1 x + a_0$

Then, we can calculate $a_1$ and $a_0$ by these equations

$a_1$=$\frac{\sum_{i=1}^{n}x_i y_i  – \sum_{i=1}^{n}x_i\sum_{i=1}^{n}y_i}{n\sum_{i=1}^{n}{x_i}^2 -(\sum_{i=1}^{n}x_i)^2 }$

$a_0$=$\bar{y}-a_1 \bar{x}$

wherer $\bar{y}=$ $mean$ $of$ $y$ and $\bar{x}=$ $mean$ $of$ $x$

Question 1: Since this method involves no iteration that is needed to compute slope and intercept in the case of Gradient Descent, isn't it better than gradient descent?

Question 2: And what is the name of this method?

Improve this question
$\endgroup$
5
  • 4
    $\begingroup$ Answer to 2: Ordinary Least Squares, aka OLS. Nobody uses gradient descent for OLS: the point is to illustrate and teach gradient descent with in this extremely well-understood setting. $\endgroup$
    – whuber ♦
    Aug 12, 2020 at 13:07
  • $\begingroup$ In multiple linear regression Andrew Ng has taught a method known as Normal Equation for solving multiple linear regression. Cant we OLS for Multiple linear regression as well and have a direct formula to derive the values ? $\endgroup$
    – Turing101
    Aug 12, 2020 at 13:10
  • 3
    $\begingroup$ As you might imagine, this is a well-discussed topic: see the top-voted posts on regression and multiple regression here. $\endgroup$
    – whuber ♦
    Aug 12, 2020 at 13:13
  • $\begingroup$ stats.stackexchange.com/questions/343069/… I have seen this link and it is saying that OLS itself is Normal Equation, I am really confused here $\endgroup$
    – Turing101
    Aug 12, 2020 at 13:24
  • 2
    $\begingroup$ See stats.stackexchange.com/search?q=regression+normal+equations. For an account to bridge your algebraic notation to the matrix notation needed for multiple regression, see stats.stackexchange.com/questions/352130. If that's not perfectly clear, see the explicit example for two regressors at stats.stackexchange.com/questions/196807. $\endgroup$
    – whuber ♦
    Aug 12, 2020 at 13:53

0

Reset to default