I have designed a variational autoencoder with 2D convolutions in the encoder and decoder. I have trained this autoenocder on 50'000 unlabelled images (64 x 80). Now, I would like to use this variational autoencoder for classification on labelled data. To achieve that, I have stacked fully connected layers on top of the encoder, i.e. the latent representation is the input to the fully connected layers. I have made the layers of the encoder trainable, so I will retrain also the autoencoder during classification.
In total I have a 30 dimensional latent space and around 1000 data points (i.e. images) in 2 classes.
I think the two most important parameters for the fully connected layers are the amount of layers and the amount of nodes per layer. I think I should choose this using random search but I need some start values or range.
What range to search for the number of layers and number of nodes per layer makes sense? Especially regarding the number of nodes I'm unsure. When my latent space is of dimension 30, should I use < 30 nodes in all layers? Should the number of nodes from layer to layer be multiplied by 2? For example, 64 nodes in first layer, 128 in second and so on.
Finally, is using Batch normalization or dropout for the fully connected layers reasonable in my situation? I have only 1000 data points and probably a very shallow networks (I think there will be no more than 4 layers).