3
$\begingroup$

Before it is pointed, I am aware that a very similar question was already asked. Still, I am in doubt regarding the concept.

More specifically, it is mentioned by the most voted answer that:

In terms of a simple rule of thumb, I'd suggest that you:

  1. Run factor analysis if you assume or wish to test a theoretical model of latent factors causing observed variables.

  2. Run principal components analysis If you want to simply reduce your correlated observed variables to a smaller set of important independent composite variables.

Question 1:

I am having difficulties on understanding based on the results I obtained from R where exactly I am inputing my theoretical model of latent factors. I am using the functions from statsmethods. On both factanal() and princomp() the inputs were the same: A table where each row represented one data point and the columns consisted of different attributes I was interested on reducing. Thus, this add to my confusion on where is this pre assumed model play its role. I noticed that for factor analysis function I used parallel analysis also suggested by the site using the nScree() function to determine the number of factors and I specified if I wanted a varimax (orthogonal) or promax (oblique) rotation. Is that what is it mean by the model? Being able to choose the amount of factors and the type of rotation?

The results being provided as visual graphs for both PCA and EFA also doesn't seem to highlight this difference which adds to my confusion. Where does this distinction can be observed on them?

PCA PCA

EFA EFA

Question 2: -- Answered

I bought a book to study about this from Richard L. Gorsuch. On this book there is something that the author caught attention on the difference between PCA (Principal Component Analysis) and EFA (Exploratory Factor Analysis): It is mentioned that PCA is for population while EFA is for sample. Is that true? I didn't see that being mentioned on any discussion I read so far. Is it irrelevant?

Question 3:

I noticed that all those methods seems to impose the normal distribution constraint. I also read that for larger sets this constraint can be ignored. Is that true or PCA, EFA and CFA are sensible to distribution constraint violations?

Question 4: Where from the results of PCA and EFA should I note that one is talking about latent factors (EFA) and the other is just clustering on components (factors) the variables? The outputs from R looks the same to me. Is it just the way I perceive what the factors being shown as output? I noted that both show me the table where I can see which I can observe which of my variables are expressed the most of my factors. What is the difference on the interpretation I should have on which variable belongs to which factor in respect to PCA and EFA? EFA is saying those with higher expression seems to be more explained by that latent factor while PCA is trying to say that factor is holding those variables from what is it observed?

Question 5 Finally the last question is regarding CFA (Confirmatory Factor Analysis).

On the same function website the following image is being shown:

Confirmatory Factor Analysis

I read that CFA is usually followed after EFA for hypothesis testing. In that sense, EFA tells you which are the latent factors (which are the output factors) and then you use CFA assuming those factors you observed from EFA for hypothesis testing?

Question 6

For EFA one of the available rotations on the literature is direct oblimium. I heard that it can accounts for both promax and varimax so 'it takes the best of two words'. Is that true? I am also trying to find a function that employs them on R, since the one suggested on the site does not. I would be happy to get any suggestion on this one.

I hope it is noted that this question is way more specific on the doubts regarding EFA and PCA and also adds to CFA so not to get closed for being repeated on the subject. If at least one of the questions is answered I am more than happy too as to clarify the confusion in my head.

Thank you.

Improve this question
$\endgroup$
2
  • $\begingroup$ possible duplicate of What are the differences between Factor Analysis and Principal Component Analysis? $\endgroup$
    – amoeba
    Commented Feb 3, 2015 at 23:17
  • 1
    $\begingroup$ I don't think this is really a duplicate. The questions here are more specific & developed than the answers there answer. OTOH, they are a broadly ranging & somewhat rambling set that aren't well answered below. My advice, if you are still interested, is to convert these into a series of focused & distinct questions that are asked in turn as the previous gets answered & your understanding improves. $\endgroup$
    – gung - Reinstate Monica
    Commented Feb 4, 2015 at 0:18

2 Answers 2

Reset to default
2
$\begingroup$

I will just address question 2. I have some doubts about how well the author knows his subject if he really said it the way you have presented it. PCA is applied to the sample just like EFA and CFA. It simply takes a list of n possibly related factors looks at how the points (samples) scatter in n-dimension space and then gets the first principal component as the linear combination that explains more of the variability in the data than any other linear combination. Then the second looks at orthogonal directions to the first to find theone out of those that explains the most of the remaining variability and so on with the 3rd and 4th. So sometimes one can take just 1-3 components to describe most of the variation in the data. That is why factor analysis and principal componet analysis are described according to 1 and 2 in your statement.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thank you. Just to make sure on your thoughts this is the quote, P. 117: "Principal factors as normally extracted are based on equations for which it is assumed that the population correlation matrix is being factored. In maximum like hood procedures it is explicitly recognized that a sample is being analyzed. Maximum like hood procedures are defined as those that best reproduce the population values (the meaning of "best reproduce" is further discussed in Chapter 7). Any factor solution that best reproduces the population values is a maximum like hood analysis. $\endgroup$
    – Oeufcoque Penteano
    Commented May 14, 2012 at 16:09
  • 1
    $\begingroup$ The terminology is bad. We don't say that maximum likelihood estimates best reproduce the population values. The maximum likelihood estimates are obtained for population parameters by selecting the values that maximize the likelihood function. This means that the exact parametric model using the fitted parameter(s), in a way best describes the observed data. Maybe this is what he intended to say but phrased it poorly. $\endgroup$
    – Michael R. Chernick
    Commented May 14, 2012 at 16:39
  • $\begingroup$ The population correlation matrix cannot be factored since it is unknown. PCA does what I described for the sample data. I don't even think that means factoring the sample correlation matrix. I am assuming he means principal components when he says "principal factors" but it could mean something else and really have something to do with the sample correlation matrix.I think you should look at other books on this topic that are much better written (e.g. books on multivariate analysis such as Gnandesikan or Jolliffee's book on PCA). $\endgroup$
    – Michael R. Chernick
    Commented May 14, 2012 at 16:46
  • $\begingroup$ Thank you Michael for answering this question 2. Hopefully I can get an answer for the others too! $\endgroup$
    – Oeufcoque Penteano
    Commented May 14, 2012 at 18:22
2
$\begingroup$

I think this article will provide answers for your questions 1 and 4:

Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4, 272–299. doi:10.1037//1082-989X.4.3.272 (PDF)

From page 275:

[...] it is important to recognize that the goal of identifying latent constructs (i.e., understanding the structure of correlations among measured variables) is different from that of data reduction. Data reduction involves taking scores on a large set of measured variables and reducing them to scores on a smaller set of composite variables that retain as much information from the original variables as possible. Data reduction does not attempt to model the structure of correlations among the original variables.

[...]

If the goal is to arrive at a parsimonious representation of the associations among measured variables, EFA can be an appropriate form of analysis. If the goal is data reduction, principal components analysis (PCA) is more appropriate.

The author than continues to elaborate on the conceptual differences differences, which will probably help to clarify things.

Improve this answer
$\endgroup$
1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.