# Detecting a given face in a database of facial images

I'm working on a little project involving the faces of twitter users via their profile pictures.

A problem I've encountered is that after I filter out all but the images that are clear portrait photos, a small but significant percentage of twitter users use a picture of Justin Bieber as their profile picture.

In order to filter them out, how can I tell programmatically whether a picture is that of Justin Bieber?

-

## migrated from stackoverflow.comFeb 14 '11 at 23:51

This question came from our site for professional and enthusiast programmers.

+1 for filtering out justin bieber –  Hans Westerbeek Feb 14 '11 at 22:42
What's your development platform? This can easily be done in .NET because it superior over all other programming environments. Simply call the Page.EradicateBieber() function. Microsoft foresaw this need and graciously provided it for us out-of-the-box in .NET 4.5. (Those of you on older versions will have to wait.) (That is, of course, all tongue-in-cheek.) –  David Stratton Feb 14 '11 at 22:46
I think I can safely assert that SO doesn't need a [justin-bieber] tag. –  skaffman Feb 14 '11 at 22:47
A Justin Bieber audio filter would be good too –  Paul R Feb 14 '11 at 22:58

A better idea might be to trash all images that appear in the feed of more than one user - no recognition needed.

-
Yeah, maybe set a threshold of 2-4 possible duplications (to handle the new-baby case) before you reject a photo. Depends on what you're going to do with the photos, I guess. –  Mark Bessey Feb 15 '11 at 1:49
Simple, elegant solution. +1. –  Robert Harvey Feb 15 '11 at 3:40
People could use different pictures of the same person. –  Rebecca Chernoff Feb 15 '11 at 3:47
(+1) at Rebecca and (-1) @ PPPPPP: This just shifts the problem. –  steffen Feb 15 '11 at 7:15
They could, but in most cases they're gonna choose from a relatively small pool of images, so it'd probably still work. Edge cases be damned - for all you know my picture is of my uncle anyway. –  naught101 Mar 21 '12 at 7:05

I have a feeling that http://www.tineye.com/commercial_api may be the solution here. Simply throw the Twitter profile image to Tineye, see if it returns images (and associated URLs) that can clearly be identified (or automatically scored using simple word-count logic) as being related to (or of) that little sack of **.

Simples!

-
Google announced the image search recently: youtube.com/watch?v=t99BfDnBZcI I don't know if it has an API yet, but that may be an alternative. –  petrichor Jun 16 '11 at 10:30

Since you are able to filter to only those that are clear portrait photos, I'm assuming you have some method of feature generation to transform the raw images into features that are useful for machine learning purposes. If that's true, you could try to train a classification algorithm (there are lots of them: neural networks, etc.) by feeding the algorithm a bunch of known Bieber photos as well as a bunch of known non-Biebers. Once you have trained the model, it could be used to predict whether a new image is Bieber or not.

This sort of supervised learning technique does require you to have data where you know the correct answer (Bieber or not), but those could probably be found from a Google image search. It also requires that you have the right sorts of features, and I don't know enough about image processing or your algorithm to know if that is a major drawback.

-
Unfortunately, the feature generation step is both the hardest and most important one :(. –  steffen Feb 15 '11 at 7:19
@steffen There is some suggestion that the OP is messing with the faces, so have some descriptor generator. –  mbq Feb 15 '11 at 12:15
@mpq: I did not doubt that, however, if the OP does not have one feature per pixel, then he has to find a meaningful aggregation level. I did not downvote, I just wanted to point to the complexity which lies behind this answer (which is, of course, correct). –  steffen Feb 15 '11 at 13:00
Right, the feature generation step is the hard part. I was assuming OP could do this since he has some mechanism for processing the images already. Even if he does though they might only be useful features for detecting face/not face instead of Bieber/not Bieber...it really depends on the features. –  Michael McGowan Feb 15 '11 at 13:17

You could use a method like eigenfaces, http://en.wikipedia.org/wiki/Eigenface. The following has a good walk through of the procedure as well as links to different implementations.

From here it is common to use this in a classification approach, train a model and then predict cases. You could do this by training on a bunch of known celebrities and if you predict a face from twitter as one in your trained model of celebrities, remove it. Similar to this http://blog.cordiner.net/2010/12/02/eigenfaces-face-recognition-matlab/

This suffers from constant amendments. Soon there will be a new Justin Bieber that wont be in your trained model, so you cant predict it. There is also a case like Whitney Houston, you may have never thought to add her before but she may be a common image out of respect and admiration for a few weeks. You will not have the downside of baby pictures as mentioned above though. To over come these problems you could use more of a hierarchical clustering approach. Removing the first few sets of clusters that are very close if they reach a certain level of support, your first cluster has 15 items before a second is constructed. Now you don't have to worry about whose in your training model but you will fall to the baby pictures issue.

-

http://face.com/ has free a facial recognition API

-

If you want to do it yourself, I would recommend using Intel's free and open source OpenCV (CV for computer vision) project.

http://opencv.willowgarage.com/

http://oreilly.com/catalog/9780596516130

-

You need to put on an algorithm detecting which person that picture is referring to. You can build a model based on different portrait pictures of famous personality and use classifiers to ensure that this picture is referring to one of your database picture. You need to use a certain classifier based on different parameters liked to the face, like distance between eyes or other parameters to rise up the accuracy of your model. There is also skin analysis. The most important is to build a good classifier. This method can be vulnerable.

But there is also a very good project working on face recognition http://opencv-code.com/Opencv_Face_Detection

-
AFAIK OpenCV and the linked site only implement face detection (ehere in the picture is a human face?) which is only a first step towards face recognition (whose face is it?) –  f3lix Nov 20 '11 at 15:48

You could try locality sensitive hashing.

-
Plain KNN is not very good for faces. Faces have been shown to lie on a ~25 dimensional nonlinear manifold of the pictures. –  bayerj May 11 '11 at 19:05

## protected by mbq♦Mar 2 '11 at 9:58

Thank you for your interest in this question. Because it has attracted low-quality answers, posting an answer now requires 10 reputation on this site.