In theory, we know while we are descending to the point where the error is zero, we give big steps that are learning rate will be big. And when we are near to the error equal to zero we start giving small steps that are learning rate will be small. But, whatever the theory is we use the same learning rate whole over the Gradient descent process. Could anyone help me to explain this?
We don't always use the same learning rate; almost the opposite in my experience. Your learning resources are probably starting off with vanilla gradient descent because it is easier to understand at first, and then knowing gradient descent sets you up to start learning about its modifications (including those that change the learning rate).
Both adaptive learning rates and learning rate schedules are quite common. I personally use Adam(tf.keras.optimizers.Adam) and Nadam (tf.keras.optimizers.Nadam) among others. And tf.keras.callbacks.LearningRateScheduler is Keras/Tensorflow callback for making your own custom learning rate schedules.
