Why does contrastive loss distinguish positive from negative samples?

Trying to learn Siamese networks for ranking tasks from here, I find it hard to understand why the contrastive loss is not symmetric for positive and for negative examples.

The contrastive loss

$$L(A, B) = y|f(A) - f(B)| + (1-y)max(0, m-|f(A) - F(B)|)$$

uses a L2 norm for positive examples, but uses a hinge loss for negative examples.

The lecture doesn't explain why that is, and Googling also didn't get anything clear.

Would love a clarification on the deliberate distinction.

• Commented May 1, 2022 at 7:55

The contrastive loss has 2 components:

• The positives should be close together, so minimize $$\| f(A) - f(B) \|$$.
• The negative portion is less obvious, but the idea is that we want negatives to be farther apart.

We can't minimize $$\| f(A) - f(B) \|$$ for all $$A,B$$, because this would just map all inputs to the same point. Instead, we want to differentiate between the classes, so we want the dissimilar classes to be far apart.

One way to move negative pairs farther apart would be to just directly maximize the distance, equivalent to minimizing $$- \| f(A) - f(B) \|$$. But if we do this, then the loss can decrease arbitrarily by moving all points farther apart.

Instead, we just want to move $$A, B$$ far enough apart, which is controlled by the margin. The loss is positive, until $$\| f(A) - f(B) \| > m$$, and then the loss is 0. That's why the loss term for comparing negatives is $$\max\left(0, m - \| f(A) - f(B) \|\right).$$ This solves the problem that the loss can decrease indefinitely, because it places a lower-bound at 0.

• but of we do this, then the loss can decrease arbitrarily by moving all points farther apart- is that a theoretical problem, or a just practical one that would cause the net not to converge? In theory, what's wrong with sending far points infinitely far from each other? Commented May 1, 2022 at 8:00
• If the loss can always be decreased, then the network will never converge! Minima occur where the gradient is 0 and the Hessian is positive definite. If both these conditions aren't met at any location, then there are no minima. In simpler terms, if you're minimizing the function $f(x) = ax+b$ or even $f(x) = -x^2$, there is no finite $x$ that is a minimum. For any finite $x$, you can choose another $x$ with a smaller value $f(x)$.
– Sycorax
Commented May 1, 2022 at 13:05
• Is there any effect on selecting m!=1? Would having any different m would this have any effect? Commented May 1, 2022 at 17:57
• Choice of $m$ just changes how widely spaced the classes' representations must be.
– Sycorax
Commented May 1, 2022 at 18:02