1
$\begingroup$

While developing deepfm model network I want to add batch norm layer because model seems to suffer from vanishing gradient. There are embedding layers, 2 layers a in deep model part and one dense that connects fm model and deep model. If I want to add batch norm which layer do I added on? What do I have to look at to judge certain layers needs batch norm?

$\endgroup$
1
  • $\begingroup$ please post a more clear representation of your scenario, with activation function, dataset type, etc $\endgroup$
    – Alberto
    Aug 16, 2022 at 10:48

1 Answer 1

1
$\begingroup$

What do I have to look at to judge certain layers needs batch norm?

If the layer inputs suffer from small/large values that jeopardize the training, then batch normalization will be useful there. For this, you need to diagnose your model by plotting the histograms of each layer's input to see where it gets out of control.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.