Let's say I have a generation task like a GAN or other method for next-frame video prediction, consisting of a 256x256x3x10 image sequence, plus a 1-hot vector of length 4.

If the result of the prediction actually depends 90% on the value of the 1-hot vector and 10% on the value of the 256x256 input, what kind of architecture should I use? Should I just append the 4 "important" values to the rest at the lowest layer, and let the network figure out the importance, or should I add that vector near the "middle" (lowest dimentional representation) level of the GAN, or should I replicate each 1-hot value to a all on or all off 256x256 "channel"?

I'm wondering if DNNs are able to generally "locate extremely important inputs" or if some artifact of normalization limits the max influence of a neuron/input and we need to make sure the inputs are fed to just the right spot in the network.


A network will figure out which neurons are 'important' and which ones aren't by modifying the weights. This is not something you have to specify on your own.

So I assume you are using convolutional and pooling layers to handle your image. At the end of these layers, you will have a fully connected layer. As the vectors don't add any information to the image data, you have to add that 1-hot vector of length 4 at the fully connected layer at the 'end' of the network.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.