Why is it a common practice to have single precision for batch normalization Even when the rest of the network is of half precision?
1
-
$\begingroup$ to reduce the effect of accumulated floating point errors on the moving statistics $\endgroup$– shimaoCommented Jun 21, 2018 at 2:02
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
to reduce the effect of accumulated floating point errors on the moving statistics – shimao