In Neural Net Training, Nowadays tanh and sigmoid activation functions in hidden layers are avoided as they tend to "saturate" easily. Meaning, if the x value plugged into tanh/sigmoid is very large or very small, The derivative at that value will tend to zero and thus the changes we derive from that gradient to the neural network will also be small.

However -- We apply changes to our NN from this gradient by multiplying it by a learning rate. If an "ideal" gradient derivative function is the weight itself (ie. a derivative of a linear function), Is there a reason people don't artificially inflate a learning rate for more saturated derivative values so those neurons are able to change easily while retaining the direction of the gradient?

In neural net training, nowadays tanh and sigmoid activation functions in hidden layers are avoided as they tend to "saturate" easily. Meaning, if the x value plugged into tanh/sigmoid is very large or very small, The derivative at that value will tend to zero and thus the changes we derive from that gradient to the neural network will also be small.

However -- We apply changes to our NN from this gradient by multiplying it by a learning rate. If an "ideal" gradient derivative function is the weight itself (ie. a derivative of a linear function), Is there a reason people don't artificially inflate a learning rate for more saturated derivative values so those neurons are able to change easily while retaining the direction of the gradient?