In linear regression, is finding the minimum of the parameters the same as gradient descent?

Question

I'm taking a course on ML and have just began.

Given a loss function, $$L = \frac{1}{N}\sum^N_{n=1}(t_n - w_0 + w_1x_n)^2$$ I am confused between the difference of using gradient descent (and maybe even its purpose) as opposed to minimizing the error by finding the best values of our parameters $w_0, w_1$ by differentiating them to find a minima for each.

$$w_0 = \bar{t} - w_1\bar{x}, \;\;\;\;\;\;w_1 = \frac{\bar{x}\bar{t} - \bar{x}t}{\bar{x}\bar{x} - \bar{x}^2}$$

Or, am I greatly confused, and this is gradient descent?

Answer 1

As it was said in the comment above, gradient is one way to find the minumum of the MSE in linear regression. One advantage of the linear regression over neural network, logistic regression... is that you have an analytical formula for coefficients which minimize your loss function. This is why you do not need to use gradient descent.

Answer 2

This is the distinction between having an objective, and the methodology you use to achieve an objective.

In fitting a model, your objective (used as a synonym here for goal, i.e. what you hope to achieve) is to minimize some loss function. In the case of linear regression, that loss function is the least squares loss, but other loss functions are used in other situation. The important point is that the process of fitting the model is synonymous with the goal of minimizing this function.

Once you have a goal, you can employ multiple techniques to achieve that goal.

First Technique: Compute the partial derivative of the loss function, then try to solve the resulting equations for the zeros of the derivative. This works, sometimes, if the algebraic form of the derivative is simple enough. Often this is just not possible, algebraically (as is the case in logistic regression).

Second Technique: Use gradient descent with some stopping criteria to approximate the zeros. This is almost always applicable, even though it does not achieve an exact solution, and allows us little leverage to analyze the properties of the solution.

So you see, both techniques attack the same goal, but they offer different pros and cons in actually achieving that goal.