What is the most effective way to populate the PCA training matrix for eigenfaces?

Question

I am experimenting with the eigenfaces algorithm, and I am unclear on a number of the finer points of the technique. For starters, consider the matrix of images used to do the initial PCA. Is it "better" to have only images of different individuals in that matrix? Or is it useful to include multiple images of the same individual?

Some of the demo implementations I have seen appear to be using only one image per individual. Some use more than one (and generally these seem to have N sample images per individual rather than varying sample sizes). The paper describing the algorithm seems to suggest that multiple images could produce more accurate results.

It would intuitively seem like multiple images of the same person would tend to weaken the PCA results. For example, if I had 100 images where 90 were of the same person, the principal components would be more about distinguishing between those 90 images of the same person than distinguishing her from the other 10 individuals. Right?

Does that same reasoning still apply when there is a uniform number of sample images per individual? If not, is there a guideline for how to balance the number of samples per individual against the total number of training images?

Answer 1

Kaelin Colclasure!

Yes, you are right, PCA works in such a way, that if you have more samples of one person, it would more likely produce eigenfaces that would look like that person. However, if you use the same number of samples for every person, you would get eigenfaces for all of them with no prior. And in this case giving more samples would give you much stable eigenfaces and even get components to describe some specific features of the images.

So I would advise to take as much samples of one person as you can, but balance the number of samples of every kind. And that applies not only for images of one person, but for man and woman, for example (if you want to deal with images of both men and women, of course).