# What does decay_steps mean in Tensorflow tf.train.exponential_decay?

I am trying to implement an exponential learning rate decay with the Adam optimizer for a LSTM. I do not want the 'staircase = true' version. The decay_steps for me feels like the number of steps that the learning rate keeps constant. But I am not sure about this and Tensorflow has not stated it in their documentation. Any help is much appreciated.

• Just so you know, Adam already handles learning rate optimization. – ARAT Jan 10 at 21:17

As mentioned in the code of the function the relation of decay_steps with decayed_learning_rate is the following:
decayed_learning_rate = learning_rate *

Hence, you should set the decay_steps proportional to the global_step of the algorithm.