1
$\begingroup$

Background: Hello, I'm creating a GAN with an RMSProp optimizer for both discriminator & generator. The generator model has half the learning rate of the discriminator (1e-4) and half the decay of the discriminator (3e-8).

Question: Keras > 2.3 no longer has the 'decay' parameter. Is the following the correct way to transfer the 'decay' parameter to Keras >2.3? I'm unsure what I would insert for "decay steps" since that wasn't a parameter before.

Keras < 2.3

RMSprop(lr=1e-4, decay=3e-8)

Keras > 2.3

lr_schedule = keras.optimizers.schedules.ExponentialDecay(
    initial_learning_rate=1e-4,
    decay_steps=10000,
    decay_rate=3e-8)
optimizer = keras.optimizers.RMSprop(learning_rate=lr_schedule)

https://www.tensorflow.org/api_docs/python/tf/keras/optimizers/schedules/LearningRateSchedule

Improve this question
$\endgroup$

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.