Purpose of L2 normalization for triplet network Triplet-based distance learning for face recognition seems very effective. I'm curious about one particular aspect of the paper. As part of finding an embedding for a face, the authors normalize the hidden units using L2 normalization, which constrains the representation to be on a hypersphere. Why is that helpful or needed?
 A: I think it's because it provides a preferred location and scale to the embedding. The preferred location means that the loss is no longer translation invariant, which is useful when you're working with floating point, while the preferred scale gives the margin parameter meaning. Without the hypersphere restriction, I think inflating the margin by a factor of $c$ would just scale all the embeddings by a factor of $\sqrt{c}$.
A: The squared Euclidean distance between normalized vectors is proportional to their cosine similarity (ref: wikipedia), 
$$\|\frac{A}{\|A\|}-\frac{B}{\|B\|}\|^2 = \|\frac{A}{\|A\|}\|^2+\|\frac{B}{\|B\|}\|^2-2\frac{A\cdot B}{\|A\|\|B\|}=2-2\frac{A\cdot B}{\|A\|\|B\|}$$
so the advantage of using normalization is more or less the advantage of cosine similarity over Euclidean distance. As mentioned in Andy Jones's answer, without normalization scaling the margin by a factor would just scale the embedding correspondingly.
Another nice property is, with such normalization the value of squared Euclidean distance is guaranteed to be within range $[0,4]$, which saves us much effort from choosing a proper margin parameter $\alpha$. 
For instance, in another paper referenced by this paper, it uses  what called the spring model which is based on (unnormalized) squared Euclidean distance, where one of the practical difficulties is in determining a proper margin and split point since the embedding constantly changes as the training proceeds.
If you're looking for implementing the normalization layer yourself, here's a blog about the derivations and implementation in Caffe (part of the blog is in Chinese but it won't affect reading).
