In the training of very deep neural nets, the vanishing gradient is the problem that the gradients of weights associated with early layers are too small leading to small overall steps in gradient descent. This seems like a valid problem if the gradient is too small to be represented using available bits. But if this is not the case, then why can't we overcome this problem by having huge learning rates (step sizes) for parameters with small gradient?
Note: This question is different from questions that have already been asked on this on stackexchange as other questions ask about role of activation function in vanishing gradient problem.