# Cant understand linear regression algorithm

I'm doing some machine learning stuff. And stumbled upon the machine learning regression algorithm. Here the derivatives are the ones from MSE:

$$f(m,b) = \frac{1}{N} \sum_{i=1}^{n} (y_i - (mx_i + b))^2$$

If you know the algorithm, jump off the code to the question. But this is an example:

def update_weights(radio, sales, weight, bias, learning_rate):
weight_deriv = 0
bias_deriv = 0

for i in range(companies):
# Calculate partial derivatives
# -2x(y - (mx + b))

# -2(y - (mx + b))
bias_deriv += -2*(sales[i] - (weight*radio[i] + bias))

# We subtract because the derivatives point in direction of steepest ascent
weight -= (weight_deriv / companies) * learning_rate
bias -= (bias_deriv / companies) * learning_rate
return weight, bias


But my trouble is with this 2 lines:

weight -= (weight_deriv / companies) * learning_rate
bias -= (bias_deriv / companies) * learning_rate


I get they are slopes in Error(weight, bias)

Why do we update the parameters like that?

Once we have the direction we need to move, why not to use

weight -= learning_step

This looked a bit complicated to me I tried $$E = X^2$$, and $$\large\frac{dE}{dX}=2X$$

So to move from a particular X to one where E is smaller I could just use X-dX so I go anywhere.

• Do you know how gradient descent works? We're just minimizing $f$ using gradient descent. At each iteration, you take a step in the direction of steepest descent, which is the negative gradient direction. You could read about the gradient, gradient descent, and how to compute the gradient of $f$. Dec 20, 2020 at 16:54
• $w -= df/dw*learning$ is wrong to me. This gives me a new point in $F$