How to use cosine similarity within triplet loss

Question

The triplet loss is defined as follows:

$$ L(A, P, N) = \max(\Vert f(A) - f(P) \Vert^2- \Vert f(A) - f(N)\Vert^2 + \textrm{margin}, 0) $$

where $A$=anchor, $P$=positive, and $N$=negative are the data samples in the loss, and $\rm margin$ is the minimum distance between the anchor and positive/negative samples.

I read somewhere that $(1 - cosine\_similarity)$ may be used instead of the $L_2\ distance$.

Note that I am using Tensorflow - and the cosine similarity loss is defined that When it is a negative number between -1 and 0, 0 indicates orthogonality and values closer to -1 indicate greater similarity. The values closer to 1 indicate greater dissimilarity. So, it is the opposite of cosine similarity metric.

Another resource I found is the cosine similarity layer here, but it is not a triplet loss.

Any suggestions on how to write my triplet loss with cosine similarity?

Edit

I am having some luck with this where I see the loss function go down

 loss = (1 - an_distance) + tf.maximum(ap_distance + self.margin, 0.0)

where ap_distance and an_distance are the cosine similarity loss (not metric - so the measure is reversed). So I wonder if the terms should be flipped.

Edit 2

Its been a while but my best advice is to use the same metric as a distance function in training and inference. For example, if you use L2 inside the triplet/ quadruplet loss - don't use cosine in inference to compare vectors.

I ended up putting the cosine distance in the triplet (actually used quadruplet) - and used the same metric in inference for comparison and worked amazing.

Here is somewhat of the sudo code:

    margin = 0.25
    loss_fn = tf.keras.losses.CosineSimilarity(axis=1)
    
    ap_distance = loss_fn(anchor, positive)
    an_distance_one = loss_fn(anchor, negative_one)
    an_distance_two = loss_fn(anchor, negative_two)
    
    ap_distance, an_distance_one, an_distance_two = siamese_network(data)
    loss_one = tf.maximum(ap_distance - an_distance_one + 0.5 * margin, 0.0)
    loss_two = tf.maximum(ap_distance - an_distance_two + margin, 0.0)
    loss = loss_one + loss_two

Answer 1

Perhaps equation 9 in this paper¹ is useful. Using your notation:

$$ \begin{equation} L_{\text{cos}}(A, P, N) = -\log \frac{\exp\{s (f(A)^T f(P) - m)\}}{\exp\{s (f(A)^T f(P) - m)\} + \exp\{s f(A)^T f(N)\}} \end{equation} $$

where

• $s > 1$ is a hyperparameter specifying the radius of the hypersphere where features live (the authors argue that increasing this allows for greater angular separation b/t features, and thus greater discrimination)
• $m$ is the margin hyperparameter (in terms of cosine distance)
• and all features were normalized to have unit norm (that's why the dot products above are the same as cosine similarity).

References

1. Unde, Amit Satish, and Renu M. Rameshan. "MOTS R-CNN: Cosine-margin-triplet loss for multi-object tracking." arXiv preprint arXiv:2102.03512 (2021).