I was studying siamese networks for authentication. Loss is:
Y is 0 for dissimilar pairs and 1 for similar pairs.
D_w is the distance (e.g. euclidean distance) between two pairs (by using weights w).
If pairs are similar, then loss is equal to the green box in loss function. If pairs are dissimilar, then loss is equal to the red box in the loss function.
Suppose that we have two similar samples, then the loss is equal to the distance^2 between them. Since the aim of the training is to minimize the loss, therefore by training on similar pairs, will help to minimize the distance between similar pairs.
If we have two dissimilar samples, therefore there is an m term, which signifies margin in the loss function. According to the red box, if margin<D_w, then, loss is equal to zero. What exactly signifies the margin. What we are separating with margin? Suppose that I want to recognize face x (aim: to understand input is face x or not), what margin does? Is it something linear (like SVM margin), circle or another form? Should it have a specific form? Why if margin<D_w, loss does nothing. I do not understand this part. I do not understand if it is a learnable parameter or a pre-defined value? If it is a pre-defined value, how do we predefine it, based on which knowledge?
Edit 1: I found this explanation but I did not understand. I put it, in case it helps you to understand: taken from here:
Role of margin m: It is to be noted that the representations of negative pairs will only contribute to the loss if the estimated distance ||f(x)-f(y)|| < m. Meaning that it will no longer care how far the negative pairs x and y are once this limit reaches. So, it can focus more on the difficult to embed points.