Doesn't Factor Analysis always overfit on a theoretical basis Imagine you have 3 items that measure some "quality", you could take for example a sum score or you can do a factor analysis.
When you use a factor analysis, don't you basically completely overfit when you assign weights and then use these weights in a model, supposing this measure IS a latent variable in a model? You're practically saying "the latent variable is some kind of "optimal" factor solution (based on the 3 variables/data), assuming it is completely captured by whatever weights are assigned by this optimal solution"
Since it is latent you cannot be sure how this variable is "actually explained" (perhaps in reality a 4th variable would be needed, or the weights are actually different, however the factor analysis just acts like "yes this is the one"), but it seems a bit weird to choose "the best possible (in terms of combined variance) solution" out of all possible ways to bring 3 variables together into a  latent variable. I'm sure this would be easy to illustrate with Venn diagrams.
I'm sorry this is so terribly written and I'm not better able to explain my worries. I'll try to rewrite it, but please bear with me until then. 
 A: This question seems to be about exploratory factor analysis (FA).  (Initially principal components analysis was also mentioned, and that method has much more to do with exploratory than with confirmatory FA.) 
I disagree that one assumes the latent variable to be "completely captured by whatever weights are assigned" to the observed variables.  Instead, one assumes that the observed variables are imperfect expressions of or indicators of that latent variable.  The extent to which they capture it or cover its domain is testable, though, through other techniques.  These might include (all:  please edit) 


*

*a variety of mostly bivariate validation methods such as those addressing convergent, construct, content, and discriminant validity;

*confirmatory factor analysis/structural equation modeling;

*regression.


Few factor-analytic solutions offer "the last word" on the nature of a latent variable.  In this very subjective art/science, there is almost always some additional var. one might include to more fully flesh out a latent var.  Or some improvement in measurement method for one of the observed vars. that will help it better express some aspect of that latent var.
(Terminology note:  "overfit" is used here to mean, I think, "assume too much."  It has a standard meaning centering on predictive models [not FA models].  This has to do with overreliance on a complex predictive model when a simpler one would do a better job of prediction with a new sample.)  
