I have a well-crafted draft of a questionnaire, containing 240 items. The average processing time is 20 secs per item, so the entire questionnaire would take around 1 hour 20 min to complete. I want to reduce the estimated processing time down to 30 minutes, to avoid fatigue among participants.

Moreover, the questionnaire should be measuring relativity stable aspects of 24 personality facets, which I in turn expect to assemble into 6 factors.

I want to find a sampling technique, that allows me to correlate all 240 items without having to give all items to every participant. This data will be used to eject items from the questionnaire, that correlate weakly. Ideally, the gathered data would be suited for factor analysis.

In short: How do I decide what items to give to which participant, to end up with a dataset where all items can be pairwise correlated, with only presenting ~ 90 items to each participant? (And how many participants would I require in such a case?)


This is an interesting, practical question. But an answer is not so easy to formulate. At least it's not clear whether a solution is practically possible.

Let's assume that the first hypothesis is correct: there are 6 factors which can be combined to form the 24 facets. I put emphasis on this, because these create a 6-dimensional vector space, where you could identify the factors as base vectors.

As far as I understand, you are trying to do two things:

  1. find questions, which have a high correlation to (the prediction of) the factors itself
  2. find questions, where you have a proper estimate on the actual value of the factor

Let's assume further, that the "well-crafted" design means that the first part is already solved.

What you hopefully have is some kind of validation data; so that when you have a correlation matrix between the input and the output. If this data is already balanced; e.g. equal numbers among each factors/facets. You can use this to extract the questions that have the smallest variation between members of the same "class", but also a difference between the members of different "classes". Mathematically you are looking for questions which answers have:

  1. high correlation to the output value/vector component
  2. small variance

If the data is representative (e.g. has a large number of entries), then you could also possibly extract "corrections" (e.g. matrices). These could be used to correct the deviation in classification for using only subsets of the initial questions. The simple test for this procedure is to verify this with the same data. Mathematically you have an input vector with 240 entries and an output vector of 6 values. Your data processing is formally the matrix, by which you multiply the input vector. You are then looking for the correction matrix, by which you need to alter your "process matrix" to obtain the same output vector.

However, if you don't have such validation data, there is very little you can do that is statistically sound to determine the effect of individual questions.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.