I am writing a facial recognition library in C++ which performs inference to generate feature vectors. When I provide my machine learning model with image data, I receive a features vector of length 512.
In some cases, I will receive multiple images of one person (perhaps from different angles, different points in time, but with no labels) and I need to generate a single feature template from these images representing the face.
Thus far, my method is to generate a feature vector for each one of these images (of the same person), then take the arithmetic mean of the features vectors to generate one template.
How should I be generating a single template? Here is my concern. Let's say we have 5 photos of a person, 1 from 10 years ago and 4 from the past 2 years. There is a good chance that 10 year old photo in the feature space is farther away from the other points. Then that 10 year old photo will be a high-leverage point for the arithmetic mean. The the distance between the computed centroid and the not so old photos will be smaller, if that 10 year old photo is not used in the mean calculation. The second concern is if the arithmetic mean is suitable for the manifold of the face points in our feature space.
Should I use some method to remove outliers (standard deviation)
Any links to academic papers I can reference would be great.