If you're using "tensorflow" and giving your model a batch make sure you put `model(batch, training=True)`. If training isn't set to true then the gradient will be all 0s.