Why is the default learning rate for Adadelta so low in Keras? I have been training a model using the Adadelta optimizer for some time, and I noticed that it converges very, very slowly. Then I checked the Keras documentation, and to my surprise the default learning rate is 0.001.
This is 1000 times smaller than the learning rate of the "real" Adadelta optimizer. When I set it to 1, my model converged significantly faster. Why has Keras chosen to set the rate so low by default?
 A: If you look into TF source code you will find
def __init__(self, learning_rate=0.001, rho=0.95, epsilon=1e-8,
               use_locking=False, name="Adadelta"):
    """Construct a new Adadelta optimizer.
    Args:
      learning_rate: A `Tensor` or a floating point value. The learning rate.
        To match the exact form in the original paper use 1.0.
      rho: A `Tensor` or a floating point value. The decay rate.
      epsilon: A `Tensor` or a floating point value.  A constant epsilon used
               to better conditioning the grad update.
      use_locking: If `True` use locks for update operations.
      name: Optional name prefix for the operations created when applying
        gradients.  Defaults to "Adadelta".
[...]

Developers are aware that the paper used learning_rate=1.0.
They put that notice there due to this issue.
With the exception of SGD, all other major optimizers have learning_rate=0.001, so it probably got the same value by coincidence.
A: It seems like Keras has enforced the same default values for each of the different optimizers. For most of the optimizers listed on this page, i.e.

*

*RMSprop

*Adam

*Adadelta

*Adagrad

*Adamax

*Nadam

*Ftrl

the default learning rate is always set to 0.001.
