# How to use cosine similarity within triplet loss

The triplet loss is defined as follows:

$$L(A, P, N) = max(‖f(A) - f(P)‖² - ‖f(A) - f(N)‖² + margin, 0)$$

where $$A$$=anchor, $$P$$=positive, and $$N$$=negative are the data samples in the loss, and $$margin$$ is the minimum distance between the anchor and positive/negative samples.

I read somewhere that $$(1 - cosine\_similarity)$$ may be used instead of the $$L2\ distance$$.

Note that I am using Tensorflow - and the cosine similarity loss is defined that When it is a negative number between -1 and 0, 0 indicates orthogonality and values closer to -1 indicate greater similarity. The values closer to 1 indicate greater dissimilarity. So, it is the opposite of cosine similarity metric.

Another resource I found is the cosine similarity layer here, but it is not a triplet loss.

Any suggestions on how to write my triplet loss with cosine similarity?

Edit

I am having some luck with this where I see the loss function go down

 loss = (1 - an_distance) + tf.maximum(ap_distance + self.margin, 0.0)


where ap_distance and an_distance are the cosine similarity loss (not metric - so the measure is reversed). So I wonder if the terms should be flipped.

• sqrt[2(1-cos_sim)] is indeed a special case of euclidean distance called chord distance. Due to the law of cosines stats.stackexchange.com/a/36158/3277. 1-cos_sim, the cosine distance, is thus like squared euclidean distance. Jul 4, 2022 at 18:27
• Cosine distance is not the only angular distance. stats.stackexchange.com/a/565057/22311
– Sycorax
Jul 4, 2022 at 19:29
• @Sycorax I am using cosine similarity metric to compare vectors after training; hence, I wanted to use the same distance during training. What do you suggest for the triplet loss. Jul 4, 2022 at 19:49

Perhaps equation 9 in this paper1 is useful. Using your notation:

$$$$L_{\text{cos}}(A, P, N) = -\log \frac{\exp\{s (f(A)^T f(P) - m)\}}{\exp\{s (f(A)^T f(P) - m)\} + \exp\{s f(A)^T f(N)\}}$$$$

where

• $$s > 1$$ is a hyperparameter specifying the radius of the hypersphere where features live (the authors argue that increasing this allows for greater angular separation b/t features, and thus greater discrimination)
• $$m$$ is the margin hyperparameter (in terms of cosine distance)
• and all features were normalized to have unit norm (that's why the dot products above are the same as cosine similarity).

## References

1. Unde, Amit Satish, and Renu M. Rameshan. "MOTS R-CNN: Cosine-margin-triplet loss for multi-object tracking." arXiv preprint arXiv:2102.03512 (2021).