I am trying to implement a simple convolutional autoencoder in theano. The architecture I have in mind is:

in --> conv --> non-linearity --> max-pool --> upsample --> deconv --> non-linearity --> out

No fully connected hidden layers yet. The weights in the conv and deconv layers are tied, so really there is only one set of parameters to learn: the weights connecting in --> conv. Currently, this isn't working; I'm quickly getting weights that go to nan very quickly.

One thing that bothers me is the asymmetry of where the non-linearities are placed. Maybe the second non-linearity layer should be placed between upsample and deconv?

Can someone offer some advice? I have read similar posts on this topic, but they don't address this issue, or the problem of weights going to nan.

Other details: I'm using the sigmoid function for the non-linearity for now. Also, I'm doing the upsampling by inserting zeros in all the unknown locations.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.