How error derivative becomes zero in gradient descent

Question

Previous questions this & this does not answer my question

Code

import matplotlib.pyplot as plt

inputs = [(0.0000, 0.0000), (0.1600, 0.1556), (0.2400, 0.3543), (0.2800, 0.3709)]
targets = [230, 555, 815, 860]

weights = [0.1, 0.2]
b = 0.3
learning_rate = 0.1

epochs = 4

# prediction
def predict(inputs):
    return sum([(w * i) for w, i in zip(weights, inputs)]) + b

# train the network
for epoch in range(epochs):
#   Feed forward--------- 
    pred = [predict(inp) for inp in inputs]
    print("Pred:", pred)
    
    
#   Back propagation------

#   error derivative
    errors_d = [(p - t) for p, t in zip(pred, targets)]
    
#   error partial derivative
    weight_d = [[(e * i) for i in (inp)] for e, inp in zip(errors_d, inputs)]
    bias_d = [(e * 1) for e in errors_d]

    weight_d_T = list(zip(*weight_d))
  
#   Update weights and bias
    for j in range(len(weights)):
        weights[j] -= learning_rate * (sum(weight_d_T[j]) / len(weight_d))
    b = b - (learning_rate * (sum(bias_d) / len(bias_d)))

From theory, In order to minimize error, we need to take the derivative with respect to the weights and bias. In the above code, I did partial derivation of the error function. And use it with learning rate and update weights and bias.

After doing some tests I figured out that if I write the weight updating equation as weights[j] -= learning_rate * (sum(weight_d_T[j]) / len(weight_d)) it will move towards down of the slope. If I write the weight updating equation as weights[j] += learning_rate * (sum(weight_d_T[j]) / len(weight_d)) I mean add partial derivative of error with previous weight value, then it will move upward of the slope. What I mean by moving upward or downward of the slope could be understood from the image below.

My question Q1: In my code, I explicitly didn't write any code on taking derivative of error equals to zero. How does the derivative reach the point where the error is zero?

Q2: After reaching the point where the error is zero or nearly zero how does the algorithm know that, it reached the point where the error becomes zero? I didn't write any if condition like if partial derivative of error == 0 then that's our target point. Without any, if statement it works. How?

I tried my best to make understand my question. But, if there is confusion please ask in the comments. Thank you.

Answer 1

Q1: weight_d is the derivative that you are calculating... the point is that is na iterative minimization procedure, thus you never set your equation equal to 0, but you slowing go down the slope, iteration per iteration... imagine you are on top a mountain, you don't jump to the lowest point, you do a step at a time where the gravity brings you

Q2:
You actually can't know, because it's an optimization algorithm... GD guarantees to converge with infinite steps... however if you have some conditions, for example strong convexity, you have an upper bound on the distance from the global minimum (for convex optimization problems)