I am experimenting with the eigenfaces algorithm, and I am unclear on a number of the finer points of the technique. For starters, consider the matrix of images used to do the initial PCA. Is it "better" to have only images of different individuals in that matrix? Or is it useful to include multiple images of the same individual?
Some of the demo implementations I have seen appear to be using only one image per individual. Some use more than one (and generally these seem to have N sample images per individual rather than varying sample sizes). The paper describing the algorithm seems to suggest that multiple images could produce more accurate results.
It would intuitively seem like multiple images of the same person would tend to weaken the PCA results. For example, if I had 100 images where 90 were of the same person, the principal components would be more about distinguishing between those 90 images of the same person than distinguishing her from the other 10 individuals. Right?
Does that same reasoning still apply when there is a uniform number of sample images per individual? If not, is there a guideline for how to balance the number of samples per individual against the total number of training images?