Are the conditions of metric space satisfied in the latent space of a classification task? Specifically, in the case of a neural network trained in a categorical classification task (cross-entropy loss function), does the final layer embedding space preserve the definition of distance between samples?
Are distances between samples in the latent space interpretable under the conditions of positivity, symmetry, and the triangle inequality? Or does the probabilistic nature of the loss function degrade metric properties in the embedding layers?
 A: This would require defining a distance metric on embedding/latent space, since there is no "built-in" distance. Assuming you pick any reasonable distance like L2, and assuming that your neural network mapping images to latent space is injective, then yes, you will also have a valid distance metric between images. If your network is not injective, then of course the distance will not be valid (if distinct two images map to the same latent vector, then their distance would be 0).
I don't think neural networks are injective because there aren't any continuous injections from $\mathbb{R}^2$ to $\mathbb{R}$, and neural networks are typically continous, so if the input dimension $m$ is greater than the embedding dimension $n$, it seems unlikely that they would injectively map $\mathbb{R}^n$ to $\mathbb{R}^m$. 
If $n \geq m$, it is possible. In fact, you may be interested in "flow-based" or "invertible" neural networks, a recent line of research exploring neural networks which specify a bijection between input and latent space.
