2
$\begingroup$

While studying GAN, I found out that ReLU activation is used at the intermediate layers, and tanh or sigmoid is used at the last layer of the generator.

I'm curious about why sigmoid or tanh is used at the last layer of the generator. Can I use ReLU activation at the last layer of the generator?

$\endgroup$
1
  • $\begingroup$ Which resource are you looking at for GANs? $\endgroup$
    – tchainzzz
    Nov 30 '20 at 2:25
2
$\begingroup$

This is due to the fact that when generating the images, they are typically normalized to be either in the range [0,1] or [-1,1]. So if you want your output images to be in [0,1] you can use a sigmoid and if you want them to be in [-1,1] you can use tanh. You can always use ReLU but you only have the garantee of it being non negative and not in a given range.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.