1
$\begingroup$

Context: In my real experiment I am planning to include a questionnaire. I aim to measure 4 different constructs with multiple questions per construct. The questionnaire now consists of 24 items. I created more questions than I need with the idea of doing a pre-test and then reducing the number of questions to 12 based on the results of factor analysis.

I was thinking of having a sample size of 30 people to participate in my pre-test, but is just what I came up with.

Question: What is the minimum sample size required for exploratory factor analysis in order to reliably refine a multi-factor questionnaire?

Improve this question
$\endgroup$
8
  • 1
    $\begingroup$ This question has already been answered. In short: 30 is very small, traditional guidelines would certainly be that you need more, at least around 100-200 but it is really difficult to know exactly as the stability of the results will also depend on the number of factors and the communalities. $\endgroup$
    – Gala
    Jul 9, 2013 at 5:56
  • $\begingroup$ Well yes, I understand that for a real questionnaire a high N is desirable. Is there no way to test the questions in a pre-test properly before using them for real? That is, to see/indicate if the questions are actually measuring the constructs you intend? $\endgroup$
    – user1747079
    Jul 9, 2013 at 6:26
  • $\begingroup$ The problem is that with a small sample the sample correlations are highly variable and the results are not stable: Items might appear to relate to another factor, etc. Either you have a stable solution and you can indeed conclude that different items are or aren't related or you just have an uninterpretable mess. In any case, my answer reflects the literature on this. $\endgroup$
    – Gala
    Jul 9, 2013 at 6:44
  • $\begingroup$ Of course, you can always do whatever you want (I have done it myself) but the fact that it's a pretest doesn't mean you're going to be OK or you won't face criticism for it later on if you try to publish your results. Sorry to be harsh but talk of “not a real questionnaire” (what is it then? fictional?) or “I don't pretend that to be scientific” (I have heard that one often elsewhere) do not change anything to the problem. $\endgroup$
    – Gala
    Jul 9, 2013 at 6:46
  • $\begingroup$ In fact, you could even argue that it's the other way around: You need a bigger sample size for an exploratory study because you have more items, probably some nuisance factors, you don't know yet if the communalities are high, etc. Validation studies for personality scales typically involve thousands of people but once you have a good scale, it's supposed to be good for very small groups (e.g. experiments) or even single measurements (e.g. personnel selection). $\endgroup$
    – Gala
    Jul 9, 2013 at 6:50

1 Answer 1

Reset to default
1
$\begingroup$

I don't think factor analysis is the way to go here at all, regardless of the number in your sample. The goal of factor analysis is to find latent variables that are linear combinations of the scores on observed variables (your questions).

The somewhat similar technique of principal component analysis is data reduction, but not through elimination of questions: Each component (like each factor) will be a linear combination of all the variables.

The idea of a pre-test to reduce the number of questions is a good one, but the way to get rid of questions is to look at correlations, item analysis, reliability, expert review and so on, not factor analysis.

Improve this answer
$\endgroup$
6
  • $\begingroup$ I disagree. Factor analysis is a pretty standard tool of that in psychology, directly using the loadings/factor matrix and indirectly to define multi-item scales on which you can conduct other type of analyses (reliability, item-whole correlation). In practice, I don't see how your suggestions would make it possible to avoid examining or assuming a factor structure first. $\endgroup$
    – Gala
    Jul 9, 2013 at 11:50
  • $\begingroup$ to find latent variables that are linear combinations of the scores on [of?] observed variables Don't quite get what you mean @Peter, but allow me to remind you that theoretically factor is not a linear combination of the variables (and component is) because of the uniqueness separated at extraction. Factor scores that are computed regressionally from the variables and are therefore their linear combinations - are distorted "good guess" factor scores. $\endgroup$
    – ttnphns
    Jul 9, 2013 at 14:04
  • $\begingroup$ Hi @ttnphns I think the factor scores still have to be linear combinations of the variables; after all, the regressions are. But the linear combination for a factor isn't as easy to get from the output as it is for a component (where it is right in the loadings matrix). After all, what else is there to put into the factor except the variables, multiplied and added? $\endgroup$
    – Peter Flom
    Jul 9, 2013 at 21:20
  • 1
    $\begingroup$ @Peter, I'll expand a bit my point (and the well known fact). One facet of the difference b/w PCA and FA is that a component is the error-free linear combination of all the variables, whereas a true factor is not. The regressionally computed factor (factor scores) is, but it is not true factor, only an approximation. Proof: correlation b/w f. scores - in contrast with c. scores - and a variable isn't exactly the loading, the true correlation. We can never compute true f. scores - because we don't know uniqueness values on the case level. $\endgroup$
    – ttnphns
    Jul 9, 2013 at 21:47
  • $\begingroup$ Hi @ttnphns OK, so, what you're saying is that the factor scores that we get out of a program aren't the real factor scores, only approximations, but those approximations are linear combinations? Also, isn't correlation of factors a matter of rotation? Or are you saying that orthogonal factors will have some non-zero correlation since they are approximations? $\endgroup$
    – Peter Flom
    Jul 9, 2013 at 21:53

Not the answer you're looking for? Browse other questions tagged or ask your own question.