3
$\begingroup$

Do I have to eliminate variables that are highly correlated before doing an exploratory factor analysis, like it has been discussed for PCA already here?

To specify, some items of my data are highly correlated r = 0.8, some items stem from a similar/partially same test [Example: Persons had to remember 20 words, they had to repeat them directly after (one item) and many minutes after (second item).] Even though this should capture different cognitive dimensions (working memory and short term memory), they are of course highly correlated. Can I use both such highly correlated items as an exploratory factor analysis? (and yes, they do load highly on the same factor). Is there a cutoff for a correlation between items that is ok?

$\endgroup$
3
  • 2
    $\begingroup$ Good question. As I've just mentioned here, whether to allow "excessively" correlated items in FA has no straightforward answer. Possibly, varied advice could be made. It hugely depends on the field of your study and the purpose of your EFA. Sure, if items are stimuli for humans then one wouldn't want to include apparent duplicates, in their eyes; in psychology, factor-developed scales are usually comprised of items similar but not too similar. Yet, again to say, it depends. $\endgroup$
    – ttnphns
    Commented Aug 8, 2016 at 11:30
  • $\begingroup$ (cont.) Try both include and exclude and compare factor structures. You might include at EFA, but exclude afterwards from a scale. Don't forget to check KMO to see if partial correlations are not strong. $\endgroup$
    – ttnphns
    Commented Aug 8, 2016 at 11:31
  • $\begingroup$ Also, you might want to turn to Alpha factor analysis to explore how well your collection of items covers the hypothesized latent trait "field". $\endgroup$
    – ttnphns
    Commented Aug 8, 2016 at 11:37

3 Answers 3

1
$\begingroup$

Opposite statements can both be true here. First off, exploratory factor analysis and even more so principal component analysis will be successful -- meaning interesting or useful -- only if there are at least some strong correlations so that some factors or components emerge as capturing large chunks of the total variability.

On the other hand, some strong correlations are banal or even side-effects of known relationships and are not instructive. This is most uncontentious when essentially similar variables are included by accident. As a silly example, it would be pointless to include temperatures measured on different scales, say Fahrenheit and Celsius, as separate variables.

Yet again, the most convincing examples are when either technique reduces a battery of closely related variables to one underlying latent variable.

These techniques have this in common: there is a complete spectrum of opinion among statistically experienced people from those who regard them as invaluable and even essential tools in their toolbox to those who regard them as elaborate ways of answering the wrong questions. A frequent area of contention is when some people want to reduce a bundle of loosely related variables to a single measure of something rather fuzzy, such as "economic development", to be used as a predictor in some model. Other modes of analysis would include all such variables in an initial model and/or of thinking hard and choosing a very few as the most appropriate predictors.

$\endgroup$
-1
$\begingroup$

The purpose of factor analysis is evaluating the relationship between observed variables.

Exploratory factor analysis is usually used to find the underline structure of the observed variables and identifying the latent structs.

PCA is mathematical tool used for finding such underline structure between variables. PCA is subjected to scaling and actual relation between variables(such as correlation).

I wouldnt linger on filtering variables pre factor analysis, mainly because the math behind most such process(PCA specifically) handles it well.

$\endgroup$
-1
$\begingroup$

The post is old but I would like to add another perspective to the topic.

You should run correlation analysis to find out high levels of association (and hence correlation) among variables.

When you perform PCA, you are trying to find an alternative model which components are orthogonal (and therefore linearly independent, with zero correlation).

At this point you must keep all the original variables, because you will be able to understand how are they influencing the new principal components and their relationship within the new model.

To summarize, with PCA you do not have to remove the old variables, you get a new set of vectors (variables) that have a linear relationship with the original dataset, defined using polynomials. The coordinates system and the "dimensions" of your original data change, but the underlying information is still there.

But you need the original dataset to understand your original variables always, you cannot remove them. Otherwise, you will lose the images of the PCA vectors in the "real world" and the knowledge about them will be lost for good. Find out about PCA rotation and you will understand how you can match the PCA model to your original dataset.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.