This may be a rather trivial question, but I am somewhat confused. I have been able to implement a convolutional variational autoencoder. I have also been able to implement a conditional variational autoencoder, though with fully connected layers only. Now, I wish to combine them, as I want to try generating images with specific attributes rather than just on a single messy latent space.
From the guides I read, the way I implemented the conditional variational autoencoder was by concatenating the original input image with an encoding of the label/attribute data when building the encoder, and doing the same to the latent space variation when building the decoder/generator. This has worked quite simply, as the layers are all fully connected.
How would I add convolutional layers into this though? For the decoder it makes sense, but for the encoder, I don't think it would make sense to concatenate the attribute data to the image then do convolutions on that. Should I do several convolutions on the input image, flatten the resulting data, then concatenate the labels, or what?
The closest description I've found is slide 70 of this presentation, but it doesn't quite make it clear enough for me.