4
$\begingroup$

I have been given a survey to analyse. There are 50 questions, and about 400 respondents. I have calculated Cronbach alpha for the entire thing, and I get about 0.9. When I do factor analysis, I do not get nice results. I need about 17 factors, according to it.

My question is: How is this possible? I would have thought that the alpha value tells me that the answers are all reasonably correlated: Someone who agrees with Q1 tends to agree with Q2, and so on. Should there then not be a small number of underlying factors?

$\endgroup$
9
$\begingroup$

Not necessarily. You might have high Cronbach's alpha while items are weakly correlated. The reason is that Cronbach's alpha does not only depend on the correlation/covariance of the items, but also the number of items. If the number of items are high enough (which is probably the case in your analysis), you will have high Cronbach's alpha, even though the items are weakly correlated. As a quick example, we can look at the formula of the standardized alpha:

$$\alpha_\text{standardized} = \frac{K \bar r}{1 + (K - 1)\bar r}$$

Where $K$ is the number of items and $\bar r$ is the mean correlation among the items. If you have 50 items, an average correlation of 0.15 will give 0.9 alpha. If this is the case, it is not surprising to get poor results from the factor analysis.


A critical assessment of Cronbach's alpha: Sijtsma, K., 2009. On the Use, the Misuse, and the Very Limited Usefulness of Cronbach’s Alpha. Psychometrika 74, 107–120.

$\endgroup$
5
$\begingroup$

You should not consider Cronbach's alpha along with exploratory Factor analysis. Alpha is from the domain of reliability, specifically, one of measures of item-item homogeneity (interchangeability) within a scale; or more specifically about alpha - how much sufficient a scale is supplied with items. Factor analysis is done to uncover latent traits and to validate a scale which pretend to be based on one factor. See reliability vs validity views.

According to classic reliability viewpoint, questionnaire items (which scores are summed or averaged to give the overall scale score) are seen as the repeated measurements of something one and same. Consider just one respondent and a scale comprised of two items. His responses are modelled as:

Item1 = T + e' and Item2 = T + e''

T stands for the "true score" of the scale. We do not claim that the true score is the value of some latent trait of factor, though we might. There could be a mixture of factors actually behind the true score; the question what is behind it is beyond the scope of reliability. Suffice to think there is something "true" that the questionnaire measures. The fact that T is the same symbol in both equations says that it is the same value in both, characterizing our concrete respondent's posession (or view) of what is being measured (we'll assume for simplicity that both items have equal so called "difficulty level"). The fact that T is capital letter serves to convey to us that within the respondent level it is a fixed quantity, not a random value which might vary.

e stands for "measurement error". It could be conceptualized as momentary fault occuring by chance while the respondent is making an answer. Items do differ by wording, however that respondent's reaction to an item peculiarity is considered by the model exactly like his unexpected sneeze to put out the pen one point away. It is single random variable within the respondent (the letter is lowercase) with two happened realizations, values e' and e''. Like most noise, they are assumed to come from normal distribution with zero mean and some variance. That variance is the measure of an individual item's unreliability. We'll assume for simplicity that variances are equal for both items, though they could differ.

Now to the essence of the question - why reliability (in the form of Cronbach's at least) may grow just simply as we add more items (as was shown in the answer by @T.E.G.). Because T (on respondent level) is constant and items are repeated measurement attempts of the same thing, varying only by the unbiased (symmetric) noise e, the more attempts (items) 1,2,3,... we use the closer the sum e'+e''+e'''+... is to zero, their expectation, and so the closer the mean of items to T. Cancelling out of noise effect on the mean as tries accumulate.

If there were no error terms (no "sneezing" effects, so to speak) then all correlations would be complete (r=1) and alpha'd be 1. It is just due to the error assumed independent from item to item that mean r come below 1 - since the simplest reliability model declares no other source of variation besides true score T and "sneezing" error e. With only 2 items, the sneeze shifts won't probably cancel each other but with many items they will. Thus, alpha grows as errors are smaller (hence correlations are bigger) and/or correlating items are more plenty.

Classic Factor analysis model is a bit more complex than the classic scale reliability model. Let us have just one-factor model:

Item1 = F + S1 + e' and Item2 = F + S2 + e''

F is the latent common factor which our scale (consisting of items) aims to measure. S1 and S2 are the specific factors belonging to item1 and item2, respectively. e is the random error term like the one described earlier.

Specific factors are not known or measured but they are real, because it is realistic to suppose that each item contains something common with other items (and what makes them to correlate) and something individual (what hinders them to correlate completely). That individual parts S1 and S2 are systematic (i.e. fixed on a given respondent level), like factor F is, and they are two different values.

Thus the "true score" of the scale T1=F+S1 and T2=F+S2 is not same magnitude in different items. Even though as you add more and more items (with individual S's) error term sum e'+e''+e'''+... will converge towards zero, it won't help stabilize on the unbiased F (our aim) as something what the scale (sum or mean of items) reflects, because we don't know S1+S2+S3... sum of values within a respondent. Adding items to a scale (construct) does not necessarily make the scale more valid as a measure of the latent factor it was intended to measure, in spite that the scale tends to grow in reliability to measure "whatever it measures", something one.

Reliability testing, to repeat, assumes that items are equivalent and their scores are just randompy noised, so adding the sufficient number of items (few, if possible, they are to correlate better, then) recovers true, unbiased score of the scale for every respondent. Factor analysis do not pose that items are equivalent, they contain true factor inside plus item-specific biases away from it, along with random noise; so summing up items (even quite well correlated due to the same factor) does not guarantee the unbiased estimate of the factor value for every respondent.

Moreover, in factor analysis as it is computationally we actually do not consider measurement errors e separately at all. That term is seen immersed into the term called "unique factor" U: U1=S1+e' and U2=S2+e'', which complements the common F in the model. Factor extraction usually will aim to maximize F variance, thus minimize on the overall Us' variances. That implies that error es' variances are also constrained, "preselected". That makes Factor analysis barely comparable with Reliability analysis. There, however, exist Alpha factor analysis which is more kindred to Reliability analysis with its Cronbach.

$\endgroup$
  • $\begingroup$ Thank you both for the anwers. I wanted to post a comment for more information, but it was too long, so I posted my own 'answer' instead. $\endgroup$ – Old_Mortality Jun 28 '17 at 22:19
0
$\begingroup$

So for example, let’s say I do a survey on healthy eating. My questions are: 1) Do you like cherry tomatoes? 2) Do you like yellow tomatoes? Whenever the respondent answers these questions, he sneezes and slightly shifts his pen on the paper. If I add the answers to the 2 questions, I average out this error. As I add more questions, I get more covariance terms in the denominator for the formula for Chronbach alpha, and –provided the covariances are positive- alpha gets bigger as I add questions. This is counteracted slightly by the K/(K-1) factor in the formula for alpha, which gets smaller as K increases. (K being the number of questions). That would also be the case even if there were no error terms (no sneezing)

Now for factor analysis: I am looking for a hidden factor (‘healthy eating’), and now we see the answer to question 1 as made up of 3 things: healthy eating, his liking for cherry tomatoes, and the sneeze. Adding more questions will average out the sneeze as before, but gets me no closer to my hidden factor, because I keep adding variables, such as his liking for yellow things, etc.

Is it necessarily a bad thing if I cannot find a small number of latent factors in a survey?

$\endgroup$
  • $\begingroup$ Not quite so as you wrote. If there were no error terms (no sneezing) then all correlations would be complete (r=1) and alpha'd be 1. It is just due to the error which is independent from item to item that mean r is below 1 - since the classic 'reliability model' poses no other source of variation besides true score T and sneezy error e. With only 2 items, the sneezy shifts won't probably cancel each other but with many items they will. Thus, alpha grows as errors are smaller (hence correlations are bigger) and/or correlating items are more plenty. $\endgroup$ – ttnphns Jun 29 '17 at 19:49
  • $\begingroup$ Is it necessarily a bad thing if I cannot find a small number of latent factors in a survey? It depends on your data and its quality for FA. The "correct" number of factors is that the smallest number whereby correlations are reproduced "sufficiently" well and the factors are interpretable after rotation. At least these two points. $\endgroup$ – ttnphns Jun 29 '17 at 19:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.