Is minimizing softmax cross entropy with two labels equivalent to minimizing sigmoid binary cross entropy with 1 label? Is using a sigmoid at the end of a neural network with 1 label equivalent to using a softmax with 2 labels?
 A: Yes. The most intuitive explanation is this: what is the difference between inferring the probabilities $p$ and $q$, where $p + q = 1$, and inferring just $p$?

Here's a more detailed explanation, if you're interested. Note that when the number of classes $K=2$, the softmax is equal to
$$softmax_1 = \frac{e^{s_1}}{e^{s_1} + e^{s_2}} = \frac{1}{1 + e^{s_2-s_1}} = \sigma(s_1 - s_2)$$
And likewise for the second class.
Hence, for a neural network, the last linear layer of which outputs the vector $s = (s_1, s_2)$ just before the softmax layer, there exists an equivalent network that has the same layers, but computes $s_2-s_1$ and then outputs the sigmoid of the result. But since $s_2-s_1$ is a linear operation, two consecutive linear layers can be squashed into one linear layer with a single output, i.e. a sigmoid network.
The same reasoning is true vice versa: a sigmoid neural network can be transformed into an equivalent softmax network by adding one more neuron with zero weights.
That's why optimizing one network is equivalent to optimizing the other.
