Dropout: when do we eliminate neurons?

Question

As you know at the training step according to the Dropout technique we eliminate every neuron with probability $p$.

The question is do we eliminate neurons on every training instance (or batch) (or something else)?

By eliminating I mean a neuron doesn't participate in the training step but it's weights are not zeroed, because it might become alive on the next instance (batch), is it correct?

I would appreciate any clarification.

Answer 1

We can train using batch dropout or per example dropout. For your second question you are correct as you do not want the weights to fit the data perfectly,you inject randomness preventing coadaptation of the weights.