In binarized neural networks, where activations are confined to -1/+1 values using the sign function, derivative of the sign function is estimated with a straight-through estimator. However, when I look at the source code for such neural networks, I see that the gradients are clipped for inputs greater than +1 and inputs smaller than -1.
What is the rationale behind doing such clipping and how are those values (-1/+1) chosen? Does it have anything to do with the fact that activations take only -1/+1 values?
Is such clipping useful for different methods of quantization (for activations and/or weights)?