my question is about a variable output or some parts of a net which can vary. For example a flag would direct which output (or some part of a net) is to choose. That means that I have different hidden layers and with means of a flag it will be decided which is to take. Is there some examples available (I have begun to work with pytorch), could you provide someone.

here a small illustration:

enter image description here

the same discussion on pytorch forum



1 Answer 1


I see two easy options:

  • Create two different optimizers, the first will optimize based on the loss at the output for the first branch of your network and will only update the weights in that part of the network. The second will do so only at the second branch of your network.
  • Use masks to decide which part of the output is taken into account in the loss (ie multiply the relevant outputs with ones and the irrelevant output with zeros). You will then only backpropagate through the relevant part of your network.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.