# Binarizing Data in a Network using the sign function

I often see the use of the sign function in machine learning models as a way to binarize data (see eqn 1 here for an example). But the derivative of the sign function is the dirac delta function, so backpropagating through the network will yield either 0 or infinity? I'm confused as to why it makes any sense to still use it?

A more concrete example:

Consider a network where each node in a hidden layer $$n_i$$ measures whether the current training point $$x$$ is within distance $$d_i$$ from some anchor point $$a_i$$. This can be represented as $$sign(||x-a_i||_2 - d_i)$$. This is a rudimentary example of a locality preserving embedding network. Naturally, the loss function will depend on the result of this node in some way. Thereofre when I try to compute the partial derivate of the loss function with respec to $$a_i$$ or $$d_i$$, it will result in a gradient of 0 and learning will not be possible.

Thanks!