6
$\begingroup$

This follows on from my previous question on assessing reliability.

I designed a questionnaire (six 5-points Likert items) to evaluate the attitude of a group of users toward a product. I would like to estimate the reliability of the questionnaire for example computing Cronbach's alpha or lambda6. So, I need to check the dimensionality of my scale. I have seen that some people use PCA to find out the number of dimensions (e.g. the principal components), other people prefer to use EFA.

  • Which is the most suitable approach?
  • Further, if I find more than one principal component or more than one latent factor does this mean that I am measuring more than one construct or several aspects of the same construct?
$\endgroup$
  • $\begingroup$ How many subjects are available? Did you observe ceiling/floor effects on your items (responses distribution shifted toward high or low values) or any possible inconsistent responses? $\endgroup$ – chl Jun 8 '11 at 14:26
11
$\begingroup$

EFA versus PCA

In a previous question on the differences between EFA and PCA, I state:

  • Principal components analysis involves extracting linear composites of observed variables.
  • Factor analysis is based on a formal model predicting observed variables from theoretical latent factors.

I find that typically within the context of developing psychological scales factor analysis is more theoretically appropriate. Latent factors are often assumed to cause the observed variables.

Assessing Scale Dimensionality

Determining the dimensionality underlying a set of likert items is not just a question of EFA versus PCA. There are multiple techniques. William Revelle has some software in R for implementing several techniques (see this discussion).

In general there is rarely a definitive answer as to how many factors are required to model a set of items. If you extract more factors, you can explain more variance in the items. Of course, just by chance you might explain some variance, so some approaches try to rule out chance (e.g., the parallel test). However, even with very large samples, where chance becomes less of an explanation, I'd expect to see systematic but small increases in variance explained by extracting more factors. Thus, you are left with the issue of how much variance must be explained by the first factor relative to others in order to conclude that the scale is sufficiently unidimensional for your purpose. Such issues are closely tied to application and broader issues of validity.

You might find the following article useful to read, for a broader discussion of definitions and approaches at quantifying unidimensionality:

Hattie, J. (1985). Methodology review: Assessing unidimensionality of tests and ltems. Applied Psychological Measurement, 9(2):139.

Here's a web presentation examining a few different decision rules for defining unidimensionality

| cite | improve this answer | |
$\endgroup$
7
$\begingroup$

Firstly, neither PCA or EFA will give you an estimate of the dimension of the scale. They are both essentially data reduction techniques. That being said, EFA is probably better for this purpose as it tells you how much of the variance in each question is accounted for in the model (the communality).

To estimate dimension, you need to use some other technique. The best ones tend to be parallel analysis, the minimum average partial criterion, and examination of the scree plot. The eigenvalues greater than one does not tend to perform well in this situation.

If you have a large amount of data, I would suggest that you take 2/3rds of it and build models. Then, fit the models you have developed to the last third of your data. This will reduce the chances of you over-fitting your data (i.e. modeling noise). This is a form of cross-validation, and is extremely important when using techniques such as factor analysis and principal components analysis, as there are many subjective decisions (factors, rotations etc) which need to be made as part of the process.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ A user has suggested making an edit to the second paragraph to indicate that dimension isn't "determined" but is estimated: the "best" value for it (in some sense) is found. I think the suggestion has merit but did not feel comfortable approving that change because your opinion on that matter is not made completely clear here. Perhaps, to resolve this issue, you might care to clarify this point? $\endgroup$ – whuber Sep 30 '11 at 3:03
  • 1
    $\begingroup$ @whuber - changed my second paragraph, it was an ill chosen word. $\endgroup$ – richiemorrisroe Sep 30 '11 at 20:53
4
$\begingroup$

Two things not mentioned so far: One: With only 6 items, you are going to have a hard time finding a lot of dimensions. Two: If you do EFA, rather than look at scree plots or eigenvalues or some other numeric test, examine several solutions and see which makes sense. Ideally, you'll be able to follow @richiemorrisroe and have a training and test sample, especially with so few items.

| cite | improve this answer | |
$\endgroup$
  • 2
    $\begingroup$ i agree with the see what makes sense approach, but I (at least) find that the numeric tests can indicate a range of possible solutions to examine, which can then be tested by both numerical means and their consistency with the measure and the aims of the study. $\endgroup$ – richiemorrisroe Sep 29 '11 at 12:05
  • 1
    $\begingroup$ I agree. But I find many people seem to want the numerical tests to "answer" the question rather than provide a range of possibilities $\endgroup$ – Peter Flom Sep 29 '11 at 13:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.