Timeline for Lack of Batch Normalization Before Last Fully Connected Layer
Current License: CC BY-SA 4.0
9 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Dec 7, 2020 at 18:00 | comment | added | OverLordGoldDragon | @HelloGoodbye Thanks; also my latest comment below question may be of interest | |
| Dec 7, 2020 at 13:47 | comment | added | HelloGoodbye | @OverLordGoldDragon I'm also wondering what class-specific dropout means, but I've run into pieces of work where they use batch normalization to do Domain-Specific Batch Normalization for Unsupervised Domain Adaptation (using the same network to process data from different domains) and to make efficient use of Adversarial Examples (to) Improve Image Recognition. | |
| Nov 14, 2019 at 18:38 | comment | added | OverLordGoldDragon | @AlexR. What is "class-specific dropout"? That'd involve adjusting masks per-sample, unsure how that'd work or whether a bias would be introduced - have a reference? | |
| Nov 14, 2019 at 18:27 | comment | added | Alex R. | I actually think this might be wrong in situations where there is class imbalance, as I’ve seen class-specific-dropout help quite well on the last layer. | |
| Nov 14, 2019 at 18:26 | comment | added | zplizzi | @OverLordGoldDragon honestly I don't remember now, although I'm pretty certain I didn't have any pooling layers in the network at all. | |
| Nov 14, 2019 at 3:23 | comment | added | OverLordGoldDragon | @zplizzi Between which exact layers had you placed BN? | |
| Sep 29, 2019 at 20:30 | comment | added | zplizzi | I can confirm that I just observed this behavior - the network would not learn with batch norm before the last layer, and worked great after removing that specific batch norm operation (but leaving batch norm elsewhere). | |
| Apr 4, 2019 at 22:37 | vote | accept | Alex R. | ||
| Nov 14, 2019 at 18:23 | |||||
| Apr 4, 2019 at 4:54 | history | answered | shimao | CC BY-SA 4.0 |