While developing deepfm model network I want to add batch norm layer because model seems to suffer from vanishing gradient. There are embedding layers, 2 layers a in deep model part and one dense that connects fm model and deep model. If I want to add batch norm which layer do I added on? What do I have to look at to judge certain layers needs batch norm?
$\begingroup$
$\endgroup$
1
-
$\begingroup$ please post a more clear representation of your scenario, with activation function, dataset type, etc $\endgroup$– AlbertoCommented Aug 16, 2022 at 10:48
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
What do I have to look at to judge certain layers needs batch norm?
If the layer inputs suffer from small/large values that jeopardize the training, then batch normalization will be useful there. For this, you need to diagnose your model by plotting the histograms of each layer's input to see where it gets out of control.
