I'm working on a toy project which I think is analogous to the problem of detecting faces and assigning names to them in Google Photos, so I've been thinking about how that process might work.
From a user perspective, I know Google Photos presents me with groups of faces which it thinks are the same person and asks me to name / label them. It later shows me new groups of faces and may ask me if these new faces are the same person as a previous group.
A naive way I could imagine this working:
- Train a model (object detection) to find faces in images, using a training set of images labeled with bounding boxes around faces
- Given an input set of images, find the faces using the model created in step 1
- Run a clustering algorithm (unsupervised learning) on the faces found in step 2, to find likely groups of the same face
- Present the groups from step 3 to the user one by one, asking for names / labels
After that, I'm shaky on what would happen next - maybe the new labels are applied to the images, and you could train a classification model using the now-labeled images to try and find other faces for that same person in new photos? But it seems like the data volume would potentially be low (since it's per user), and it seems expensive to train a customized model per user. Or maybe nothing happens next, and it's simply "find faces, cluster them, get the user to label en-masse by presenting clusters to the user".
I don't necessarily expect anyone to know exactly how the system works, but any pointers on areas to research, or educated guesses, would be interesting.