I am learning about sampling methods for Deep Embedding Learning. I was reading an article named: "Sampling Matters in Deep Embedding Learning" (https://arxiv.org/abs/1706.07567).
In the following paragraph the authors explain why sampling (too) hard negative samples for triplet loss training leads to bad results.
They show that the gradient of the loss with respect to negative samples depends on $||h_{an}||$ (the distance between the anchor and the negative sample). When this value is small it becomes sensitive to small amounts of noise. What I don't understand is why do we care about the gradient of the loss with respect to the input? when we train the network we update the weights with the gradient of the loss with respect to the weights. What am I missing here?
Thanks in advance!
