This may be a rather trivial question, but I am somewhat confused. I have been able to implement a convolutional variational autoencoder. I have also been able to implement a conditional variational autoencoder, though with fully connected layers only. Now, I wish to combine them, as I want to try generating images with specific attributes rather than just on a single messy latent space.

From the guides I read, the way I implemented the conditional variational autoencoder was by concatenating the original input image with an encoding of the label/attribute data when building the encoder, and doing the same to the latent space variation when building the decoder/generator. This has worked quite simply, as the layers are all fully connected.

How would I add convolutional layers into this though? For the decoder it makes sense, but for the encoder, I don't think it would make sense to concatenate the attribute data to the image then do convolutions on that. Should I do several convolutions on the input image, flatten the resulting data, then concatenate the labels, or what?

The closest description I've found is slide 70 of this presentation, but it doesn't quite make it clear enough for me.

  • $\begingroup$ I am trying to implement similar thing too. Accroding to some approches from gan, I think its possible to use cnn for image, and then concat with label, end up with a fully connected layer. I will atach my github repository here after completion. $\endgroup$ – Jk Rong Apr 25 at 12:39
  • $\begingroup$ sites.google.com/site/attribute2image I believe this paper can help you understand this problem $\endgroup$ – Jk Rong Apr 25 at 12:57
  • $\begingroup$ @JkRong Thanks for the help, I have successfully replicated large parts of the model described in the paper using Keras. $\endgroup$ – Ethan Qu May 15 at 3:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.