6
$\begingroup$

I am currently researching in the facerecognition field.

And I can not understand how the facenet algorithm handels a new image

They use an euclidean space for image representation. Which means that the elements in this space represent the images (faces). To create this space they use triplet's of images (1 Anchor, 1 Image of the same person (ancher) and a foreign image). These triplets are generated via a data-mining method.

In the end result images of the same person are close together and seperated via a margin. Which means that all images outside this margin belong to a foreign person. So similarity is measured by distance.

If we now want to recognize or verify a new image/face, what exactly is done?

It seems we will need again triplets, so a single image is not even possible? And what happens then? This triplet must be compared somehow again to all existing triplets to find a place for this new image in the euclidean space? If it is whithin a margin that this new image got verified, otherwise its unkown?

$\endgroup$
2
  • 1
    $\begingroup$ The basic idea is to compute the feature vector for your image as provided by the prediction function of the neural net. If you have similar information for a reference set of images, just find the image(s) in that reference set that are closest to your image regarding these features (nearest neighbour). $\endgroup$
    – Michael M
    Oct 21, 2016 at 5:50
  • $\begingroup$ without reading the paper, wouldn't recognition just be, compute similarity measure and if it's further away than the margin then it's a foreign image? Is there anything wrong with that solution? $\endgroup$ May 26, 2019 at 20:16

1 Answer 1

4
$\begingroup$

After the network is trained, we can throw away the loss layer. Actually the facenet (and many other networks for facial recognition) is trained for extracting features, that is to represent the image by a fixed length vector (embedding).

The triplet loss basically says, the distance between feature vectors of the same person should be small, and the distance between different persons should be large.

enter image description here

After training, for each given image, we take the output of the second last layer as its feature vector. Thereafter we can do verification (to tell whether two images are of the same person) or clustering based on the features and some distance function (e.g. Euclidean distance).

$\endgroup$
6
  • $\begingroup$ So basicly verification can be only done with images, that are part of the dataset. It will not work with images that are not in the dataset? $\endgroup$
    – Roman
    Oct 25, 2016 at 10:01
  • $\begingroup$ @Roman no, after the network is trained, we can feed any image $x$ to the network and get the output of the L2 layer $v$ as its feature vector. so in order to compare the similarity of two images $x_1$ and $x_2$ we can just compare $v_1$ and $v_2$. we don't need the triplet layer anymore after it is trained. $\endgroup$
    – dontloo
    Oct 25, 2016 at 10:22
  • $\begingroup$ Ah okay, now I have an idea what happens. Right at beginning all images are turned to vectors with values by using the NN to extract features. And the triplets are used to organize the given dataset, so the distances are adjusted. $\endgroup$
    – Roman
    Oct 25, 2016 at 15:14
  • 1
    $\begingroup$ @Roman yes, the triplets are only used in training to select some proper samples for the loss function. you can take a look at this paper machinelearning.wustl.edu/mlpapers/paper_files/…, it's one of the first that uses distance between pairs as part of the loss for face verification tasks. $\endgroup$
    – dontloo
    Oct 25, 2016 at 15:31
  • 1
    $\begingroup$ @dontloo The paper you linked to does not exist. Can you please re-link it, or perhaps provide the name of the paper? $\endgroup$ May 23, 2018 at 13:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.