Is there a reason to leave an exploratory factor analysis solution unrotated? Are there any reasons to not rotate an exploratory factor analysis solution?
It's easy to find discussions comparing orthogonal solutions with oblique solutions, and I think I completely understand all of that stuff. Also, from what I've been able to find in textbooks, the authors usually go right from explaining the factor analysis estimation methods into explaining how rotation works and what some different options are. What I haven't seen is a discussion of whether or not to rotate in the first place.
As a bonus, I'd be especially grateful if anyone could supply an argument against rotation of any type that would be valid for multiple methods of estimating the factors (e.g., principal component method and maximum likelihood method).
 A: Yes, there may be a reason to withdraw from rotation in factor analysis. That reason is actually similar to why we usually do not rotate principal components in PCA (i.e. when we use it primarily for dimensionality reduction and not to model latent traits).
After extraction, factors (or components) are orthogonal$^1$ and are usually output in descending order of their variances (column sum-of-squares of the loadings). The 1st factor thus dominates. Junior factors statistically explain what the 1st one leaves unexplained. Often that factor loads quite highly on all the variables, and that means that it is responsible for the background correlatedness among the variables. Such 1st factor is sometimes called general factor or g-factor. It is considered responsible for the fact that positive correlations prevail in psychometrics.
If you are interested in exploring that factor rather than disregard it and let it dissolve behind the simple structure, don't rotate the extracted factors. You may even partial out the effect of general factor from the correlations and proceed to factor-analyze the residual correlations.

$^1$ The difference between extraction factor/component solution, on one hand, and that solution after its rotation (orthogonal or oblique), on the other hand, is that - the extracted loading matrix $\bf A$ has orthogonal (or nearly orthogonal, for some methods of extraction) columns: $\bf A'A$ is diagonal; in other words, the loadings reside in the "principle axis structure". After rotation - even a rotation preserving orthogonality of factors/components, such as varimax - the orthogonality of loadings is lost: "principle axis structure" is abandoned for "simple structure". Principal axis structure allows to sort out among the factors/components as "more principal" or "less principal" (and the 1st column of $\bf A$ being the most general component of all), while in simple structure equal importance of all the rotated factors/components is assumed - logically speaking, you cannot select them after the rotation: accept all of them (Pt 2 here). See picture here displaying loadings before rotation and after varimax rotation.
A: I think this might help you: https://www.utdallas.edu/~herve/Abdi-rotations-pretty.pdf
Regards,
