Upon using the dropout technique, my cost function is spiking arbitrarily. Is this normal? If not, how do I avoid it?
I'm using a salt-and-pepper mask to drop out neurons at a dropout rate of 5%. I've scaled the weights while training, by a factor of 1-dropout rate.