Numerical computation of cross entropy in practice The equation for cross-entropy is:
$H(p,q)=-\sum_x{p(x)\log{q(x)}}$
When working with a binary classification problem, the ground truth is often provided to us as binary (i.e. 1's and 0's).
If I assume $q$ is the ground truth, and $p$ are my predicted probabilities, I can get the following for examples where the true label is 0:
$\log\; 0 = -\inf$
How is this handled in practice in e.g. TensorFlow or PyTorch? (for both the forward pass and the backward pass)
 A: Exponentials of very small numbers can under flow to 0, leading to $\log(0)$. But this will never happen if you work on the logit scale. So, use logits. The algebra is tedious but you can rewrite cross entropy loss with softmax/sigmoid loss as an expression of logits. Elements of Statistical Learning does this in its discussion of binary logistic regression (section 4.4.1, p. 120).
Suppose your network has 1 output neuron that gives any real number $z$ as an output. We can interpret this number as the logit of the probability that $y=1$. The probability that $y=1$ given the logit is $\Pr(y=1)=\frac{1}{1+\exp(-z)}$ and likewise $\Pr(y=0)=\frac{\exp(-z)}{1+\exp(-z)}$.
Combining this expressions with the formula for binary cross entropy and doing some tedious algebra, we find
$$\begin{align}
H&=-y\log(\Pr(y=1))-(1-y)\log(\Pr(y=0))\\
&=-yz+\log\left(1+\exp(z)\right).
\end{align}$$
This means you'll never worry about $\log(0)$ because the logarithm always takes a positive argument. We know $\exp(z)>0$ because $z \in \mathbb{R}$. Positive numbers are closed under addition, so $\log(1+\exp(z)) > 0$.
Numerically, we might be concerned about overflow from $\exp(z)$. This is easily avoided if we replace the softmax function $f(x)=\log(1+\exp(x))$ with the approximation
$$
f(x) = \begin{cases}\log(1+\exp(x)) & x \le c \\
x & x > c\end{cases}
$$
as $f$ is well-approximated as the identity function when $x$ is large. Choosing $c=20$ is typical, but it might need to be larger or smaller depending on the floating point precision.
