5
$\begingroup$

Suppose I have a psychological questionnaire asking 30 questions about a person's mental health (on a Likert-scale 1-7). These 30 questions fall into 7 separate, but correlated categories.

The questionnaire has been used for several years, but I would like to develop a shorter version of it for better respondent experience (and to simply reduce survey length)

Suppose I have data collected from around 1000 individuals, and would like to use that information to reduce survey length.

My goal is to remove questions from the survey that might be adding to the survey length but not providing any additional information.

I read that Factor Analysis can be used in cases like this, but am not sure how to apply it in my scenario.

What would the steps look like using Factor Analysis for removing redundant questions from the questionnaire to shorten the survey?

In simple terms, how can Factor Analysis be used to remove questions from my survey without losing relevant information? Would you be able to provide a reference to a book/website that explains Factor Analysis for this purpose?

Here's my understanding so far, but I am not sure these are the right steps or if I am missing anything:

  1. Run Exploratory Factor Analysis on data to identify factor patterns (would CFA be more appropriate here?)
  2. Remove items that do not load on any factors

How can identify if a question is not needed? For example, if it is almost the same as another question on the survey.

$\endgroup$
4
  • 1
    $\begingroup$ Thanks for sticking with this. I think this is a good question now (+1). Hopefully, you will be able to get all the help you need. $\endgroup$ Apr 23, 2019 at 1:18
  • 1
    $\begingroup$ Agree with @gung that this is a good question. One clarification: are you adamant about using factor analysis or are you open to an alternative approach (with some similarities)? Item Response Theory (IRT) models would be very useful for this purpose and I could create an answer around that if it would be useful for you. $\endgroup$
    – jsakaluk
    Apr 24, 2019 at 18:46
  • $\begingroup$ @jsakaluk thanks for your answer - I am not glued to factor analysis, it is just what I have encountered the most when doing some research on the topic (especially used in psychology). Anything that might address my goal would be highly appreciated! (IRT or any other model - note that I'm not very familiar with IRT) $\endgroup$
    – Grint
    Apr 25, 2019 at 20:36
  • $\begingroup$ @jsakaluk hi! An answer would be really appreciated if possible $\endgroup$
    – Grint
    May 1, 2019 at 14:28

2 Answers 2

1
$\begingroup$

If you are doing the analysis for serious research projects, I would recommend not to remove any questions. As you said, "The questionnaire has been used for several years." That means they seem to work pretty well previously. Unless you have evidence that these questions are not valid measurement items, keeping them would be better. Another reason is that you have 30 questions falling into 7 major categories. That is to say, on average each category (I would like to call it a subconstruct) only has around 4 questions/items. Removing questions from your current pool might lead to some subconstruct having not sufficient number of items to accurately measure them. Generally it is hard to accurately measure a construct with just 2 or 3 items (usually Cronbach's alpha would be low). You need to take into account how the question removal would impact your measurement model. I would say having a valid measurement model is much more important than reducing the number of questions.

I only recommend you to remove some subconstructs (categories) if they are not very relevant to your analysis. Not all the categories are equal. Only keep those that are important to your study. In this way, you can remove questions associated with those unimportant categories and at the same time still be able to build valid measurement models for those important categories.

Technically speaking, you can conduct a pilot study to collect data with all questions. If you already collected the full data in a previous study, it would be fine to reuse the old data. Then use factor analysis in statistics packages such as R to see if some questions are not highly loaded to their categories. For an example, refer to https://www.statmethods.net/advstats/factor.html. For those questions having small leadings (< 0.4), you can remove them from your measurement model. Once you fit a good measurement model without these questions, you have a smaller set of questions that can be used for your final study.

$\endgroup$
0
$\begingroup$

Rather than remove questions that do not load on any factor, you might remove some of questions loading on the same factor. The idea is that such questions are highly correlated, and hence provide partly redundant information.

On the other hand, questions that do not load on any factors are highly specific questions, unlike any other, and should be retained if you deem the answer informative for the issue of interest.

$\endgroup$
3
  • $\begingroup$ Thanks. Do you think EFA or CFA is appropriate in this case? This is because I already have the theoretical constructs each question belongs to. How can I decide which questions to remove that load on the same factor? $\endgroup$
    – Grint
    May 9, 2019 at 17:16
  • $\begingroup$ I am not sure about EFA or CFA. The choice of which questions to remove I believe is highly problem-dependent. I think a bit of trial and error will show you which questions can be removed with minimal degradation of the amount of variance accounted for by the common factors. $\endgroup$
    – F. Tusell
    May 9, 2019 at 18:07
  • $\begingroup$ Thanks again, could you expand on some ways to try this? (when you say "trial and error") $\endgroup$
    – Grint
    May 9, 2019 at 19:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.