As I found in some tutorials, they didn't perform BN on last layer. It seems like a best practice, but I didn't find any detailed explanation of why this helps training.
Can anyone kindly help me brief the reason for why doing this?
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
On a final layer you don’t want to make any transformations beyond those needed to transform your outputs to appropriate form, i.e. change shape, for classification scale to unit range, or -1 to +1, for regression not use any activation, or use something as ReLU, or exponential function if the outputs need to be non-negative etc.
