Current discussions here in SSE made me to reconsider the PCA and FA models and procedures. I got curious how one would determine confidence intervals for the components/factor loadings by assuming the structure in your samples correlations is a predictor for the populations correlations and even more:
if you assume that in the population with some measured items there is a true factor structure latent which reflects varimax-, pc-, paf-models (or any theoretical structure in the loadings) and in your sample you try to reproduce that models/structure - can one assume some confidence intervals for the estimates for the population?
What I've tried was so far to find such confidence intervals by bootstrapping; for instance assume some synthetical data of four items for the population and a model of pc/factor loadings, then draw samples with smaller N and look how the pc/factor loadings reflect the "true" (population's) loadings and getting from this confidence intervals.
I do not yet know whether such an approach is meaningful at all, but just produced some results for models of pc, varimax-rotated pc, "little jiffy" (see end of posting), and paf with 2 factors. I took a set of S=256 samples each of N=40 and another set of N=160 each. Here are a couple of scatter plots of the pc-loadings of the first two pc's the same for the varimax and for the paf models.
Q: is such an approach meaningful at all? And if, how could I proceed? (The confidence intervals seem to be either elliptical or circular or twovariate normal or ???)
Remark: while I'm writing this I recall, that an answer likely exists in SEM-methodology, but until now I didn't look in this deeper because of the heavy machinery in some papers which I've tried to read... So I tried for the beginning some intuition by my described experiments.
Here are some pictures from my website where I have some more images and more explanations of the procedure and of observations.
The colours of the clouds indicate the four different items; the white circles in the clouds indicate the factor loadings in the population (according to the referred modeling)
model pc , axes (ax1,ax2), n=40
model pc , axes (ax3,ax4), n=40
model pc , axes (ax1,ax2), n=160
model varimax , axes (ax1,ax2), n=160
model paf (2 factor) , axes (ax1,ax2), n=40
model paf (2 factors) , axes (ax3,ax4) = (pc1,pc2) of the residual covariance, n=40
Here is an excerpt of an article of H.Kaiser describing how the term "little jiffy" was coined: