have been reading up here at Cross Validated (much appreciated for all the thoughtful content here😎) and got some questions about factor analysis,
i have mostly read this article on this subject, https://stats.idre.ucla.edu/spss/seminars/efa-spss/#:~:text=The%20elements%20of%20the%20Component,thought%20of%20as%20R2%20.
1, is the gist of factor analysis basically finding the best few variables that explain the bulk of the rest? (e.g., say finding two factors that explain ten variables)
2, if this is the goal, can we just simply use the variable that has the highest average absolute correlation with all the rest variables to arrive at roughly the best existing variable ? (e.g., in the correlations table in the article aforementioned, simply average the absolute values in the first column "statistics makes me cry", which gives us 0.3038)
3, further, can we about correctly say that eigenvalues are basically the equivalent of r square in linear regression, and eigenvectors are basically the coefficients? if yes, can we just simply use coefficient * r to find the "linear regression" version of loadings?
4, my biggest question is, just how does SPSS or any other software find the "factors"? from what i read PCA "conjures up" factors based on linear combinations of existing factors, and CFA finds factors that are the common factors of existing factors, both methods present "unobserved factors". just how exactly is this done? (more specifically i think it is the "Extraction Sums of Squared Loadings" in the article quoted that i am referring to)
i am relatively new to statistics (i studied economics/finance in school, came across some statistics but never really understood the material), my apologies if my questions were absurd/stupid:)
thanks in advance for your great insight or any other comments🙏