0
$\begingroup$

I have administered a questionnaire to respondents asking about customers' adoption of technology. I want to do an exploratory principal components factor analysis in Stata to form constructs from a couple of questions (in line with the existing literature).

The variables that I want to construct are perceived usefulness, perceived ease of use, perceived risk with product (PRP), etc. and I am comparing customers from two countries: The Netherlands (n=63) and Bulgaria (n=101).

I designed items based on the research literature. So I expected that there would be no problems with the factor analysis; i.e., all items would have high factor loadings on a single factor. I have obtained cronbach's alpha which is 0.7 for all factors and 0.6 for usefulness for the Dutch customers, which is OK. The factor analysis is fine for most of the factors and most of items load most on one single factor as suggested by literature.

The problem is that for PRP I got 4/5 factors for Bulgarians and 6/7 factors for Dutch which are actually different constructs compared to PRP for Bulgarians (13 factor loadings for PRP). I also have similar problem with Subjective norm (SN).

My questions:

  • When items do not fit into one factor is it possible to just take the means of such items? *
  • Or is there something else I should do?
  • Can you also provide me with some reference?
$\endgroup$
3
$\begingroup$

As a general advice, your sample size is quite small. It's not such a no-no as some people claim but depending on the specifics of the data, it's not too surprising to have unstable or unexpected results in a factor analysis.

A big question in all this is how you selected the number of factors to extract. There is no objective easy-to-determine “number of factors” in the data, so when you are saying “I have got 4/5 factors” there could be a lot of reasons for that. The default choice in some software packages (e.g. SPSS) have been heavily criticized.

If after reading up on that and adjusting the analysis, you are still unable to recover a reasonable structure, what you should do next depends mostly on the source of your scale:

  • If this is an ad hoc scale that you created yourself, it is not surprising at all that some items do not perform as expected even if you formulated them based on a well-established literature. Ideally, you should collect a larger sample and try to refine the scale. This is a large and complex topic so it's difficult to give specific advice without a more focused question but it is common to start with a large set of items and reduce it using factor analysis. You could therefore just focus on the most interpretable factors and select the items that seem most cleanly related to them, throwing the rest out. Of course, these shorter scales will have a lower reliability than if all the items worked as well as intended.

  • If this is a translation of a well-known scale, this is worrisome and there is no easy solution. Ideally, you would review the translation item-by-item and collect a much larger sample size for a validation study (perhaps with some confirmatory factor analysis). Again, this is a large and complex topic so it's difficult to give more specific advice.

  • If this is well-known questionnaire that has already been used in Dutch and Bulgarian samples, you could just ignore the EFA results and go on with your analysis (you could in fact argue that there was no point in trying an exploratory factor analysis of a well-known scale with such a tiny sample in the first place).

Also, you could try to look at the specific item loadings on each factor. Intuitively, a messy structure with items “switching” factors is more of a problem than factors split in half with all items supposed to measure a construct ultimately falling together in two or more subfactors.

$\endgroup$
  • $\begingroup$ Thank you for your answers. I have included new items in my questionnaire and somewhat changed some of the other items. I suppose this was not wise. I got better results when I take the means for perceived risk with product and subjective norm instead of extracting the first factor and use factors for the others. That is why I am asking if it is possible to combine in the regression means and factors as explanatory variables. The papers which deal on a similar topic do not have problems with the items and use SEM or regression analysis. $\endgroup$ – Badan May 16 '13 at 11:54
  • $\begingroup$ I am not sure I understand what you did or intend to do. Also, in what ways are your result “better”? One problem in simply lumping different items together is one of interpretation: If they are not correlated with each other in the first place and do not reflect the same factor, it's difficult to make sense of any relationship you might find between your scale and other variables. $\endgroup$ – Gala May 17 '13 at 11:17
1
$\begingroup$

What comes first here? Understanding the data or being committed to using a particular technique? Why precisely do you think the PCA is going to help?

Perhaps it is beyond your control, or too late any way, but the questionnaire sounds badly designed. If a rating of "perceived usefulness" is needed, why not ask something like "on a scale of 1 to 5, please rate how useful this thing is"? (Similar comments for other scales.)

More positively, I would

  1. pool the data for two countries, and then see how the people from each country compare on the scales defined by the PCs. The different sample sizes don't seem especially worrying here.

  2. do a PCA for Netherlands, then see (as above).

  3. do a PCA for Bulgaria, then see (as above).

  4. see which of #1 to #3 was most illuminating.

I wouldn't try anything more complicated. What you have in mind sounds like an analysis where you would have a hard time disentangling side-effects of what you did from something revealing the structure of the data.

I am calling what you did PCA, on the philosophy of calling a spade a spade (you may need a different idiom in your first language).

$\endgroup$
  • 1
    $\begingroup$ Multiple items scales are standard practice in psychology, analyzing them with factor analysis (and confusing PCA for a factor analysis technique) too. Of course, we could have a discussion on the merits of this approach but it's easy to see where the OP is coming from and not too helpful to say “you should just ignore all this and use single items”. $\endgroup$ – Gala May 16 '13 at 10:18
  • $\begingroup$ The statement in quotation marks is not to be found in my answer, nor is it what I suggested. My positive suggestions include PCAs. Reversing your first statement, I have encountered many instances in which people are using supposedly standard methods that get in the way of helpful analyses. If people have a good reason for using a particular method, that's fine. If they can't provide a good defence, they should think more about their goals. $\endgroup$ – Nick Cox May 16 '13 at 10:44
  • $\begingroup$ What does “the questionnaire sounds badly designed” and “why not ask something like "on a scale of 1 to 5, please rate how useful this thing is"?” mean, then? Without even acknowledging that this is definitely not the way psychological scales are usually designed and without any reference or specific criticism of this standard practice, it does not seem very enlightening. $\endgroup$ – Gala May 16 '13 at 11:04
  • $\begingroup$ I do not mean to suggest that using “standard methods” is necessarily a good idea or that psychologists are right to use factor analysis in the way they do but having a good defence seems even more important when departing from them. $\endgroup$ – Gala May 16 '13 at 11:16
  • $\begingroup$ I stand by my comments. I think we have a minor culture clash here. $\endgroup$ – Nick Cox May 16 '13 at 11:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.