6
$\begingroup$

I have designed a rather long (250 Qn) survey designed to uncover user clusters. The questions are such that the pattern of answering should elicit user clusters, but I am having trouble uncovering these with my analyses to date.

For example, a typical Qn might be: 'Are you more of a dog or cat person?' Or, 'Chocolate or vanilla?' etc.

I'm coding these questions in a binary format. So, if the user answered the two questions above with 1. Dog and 2. Vanilla, the user's answer matrix would look like: [1 0 0 1] Signifying that the user chose the first and fourth answer, where the answer space is [Dog Cat Chocolate Vanilla]

I have roughly 300 respondents who have answered all 250 questions, giving around 800 possible answers, so my binary [user x answer] matrix is 300 x 800.

I have run SVD on this matrix. The first factor relates to the number of people who selected that answer (magnitude) as expected. The second factor clusters nicely into male / female (I know because I ask gender) respondents.

My problem is, all other factors are Gaussian and offer no way for me to split them into groups. A plot-matrix of the factors shows no grouping whatsoever. A clue: when I look at the highest and lowest factor values for factors 3, 4, & 5, I can determine that there are definite personality types represented. For example, cautious/risky or conservative/outgoing or frugal/outlandish. But these are just the tails of a Gaussian. I am completely unable to separate these questions by anything but a 'random' threshold to the Gaussian tail.

The goal is to have a subset of the 250 questions that would allow me to quickly characterize a respondent, but right now, the only clustering I am able to assign is that of gender.

$\endgroup$
7
  • 6
    $\begingroup$ I think you're describing two related, but different questions here - 1) how does your respondent space group together to find cluster of participants who behave in a similar fashion? This is primarily addressed through a latent cluster analysis technique (of which there are many). 2) Can you reduce the attribute space from 250 questions down to a smaller number of questions? This is primarily done through a factor or principle component analysis. One of the key outputs to this is how your observed factors "load" onto the unobserved factors. This sometimes is used as an input into clustering. $\endgroup$
    – Chase
    Commented Apr 13, 2012 at 18:25
  • $\begingroup$ @chase, thanks for your reply. I've been working on both fronts. Perhaps you can explain what you mean by "observed factors "load" onto the unobserved factors". I'm currently looking at the questions corresponding to factors with the highest and lowest values. These give me opposites in latent space. I can tell because the questions clearly indicate opposition within the users. Even with this opposition, no real clusters reveal themselves. The distribution is too 'continuous'. $\endgroup$
    – zbinsd
    Commented Apr 13, 2012 at 23:02
  • $\begingroup$ can you simplify the input before running your clusters? i.e. if they are continuous responses, maybe turn them into binary variables at some threshold? Even if that's not possible, you can try removing the variables that contribute the least to the clustering solution. In a recent project, I started with 28 variables and ended up putting a cluster solution based off 8 of them. The others really just added noise and did not differentiate between the groups...i.e they were important to all respondents, or none of respondents, etc.. $\endgroup$
    – Chase
    Commented Apr 13, 2012 at 23:20
  • $\begingroup$ @chase. All the data are as described above. For ex. if a q'n had 3 answers and the user chose answer #2, then she would receive [0 1 0]. I'm wondering if this is the best coding scheme for a survey. None of the q'n have continuous or range values. They're all 'do you like this or that' type of q'ns. I did what you describe. I tried keeping only the q'ns that corresponded to the top and bottom three loadings for the 3rd-5th factors. That gave me 3*2*3 = 18 question answers to cluster on, but all 2D plots are just Gaussian clouds. NO clusters! :) $\endgroup$
    – zbinsd
    Commented Apr 14, 2012 at 0:05
  • $\begingroup$ Using the method I describe, I completely disregard the 'QUESTION' and use only the 'ANSWER' with the largest factor loadings. I'm wondering if this is short sited. $\endgroup$
    – zbinsd
    Commented Apr 14, 2012 at 0:06

3 Answers 3

5
$\begingroup$

Just addressing the coding part of the question, building on the comments. Most stats packages are set up to deal with a coding like

What operating system are you using?

1 = Mac
2 = PC
3 = Linux

with a flag on that variable that it is a categorical factor and hence the 1, 2, 3 should not be interpreted as a continuous variable but just as the form of coding. This approach can also nicely accommodate a coding for NA.

This way of coding is useful because you can do a lot of categorical data analysis that involves matching one variable against another eg in a contingency table or (getting a bit fancier) correspondence analysis. This is easy to get the stats package to do with the above set of coding but requires a fair bit of mucking about in your approach.

Then, in much subsequent analysis, you need to specify some kind of contrasts to get it back to a set of binary variables similar (but not identical) to what you have started with. Commonly, one level of each variable is set as the "corner point" (eg "Mac") and the other levels of each categorical variable get their own binary variable in the new coding. The idea being that each row of data is assumed to be a Mac unless they have a 1 in the PC or Linux columns.

It's not uncommon for machine-read survey data to come in with a set of binary variables such as you have, but then typically the analyst starts by converting them to multi-level categorical factors such as I describe above. And a subsequent conversion back to multiple variables of 0s and 1s is typically done under the hood by the stats package if you ask it to do eg cluster analysis - so long as the stats package understands that these are categorical variables and hence treats them accordingly.

So the main advantage of this approach is that most stats packages are built around such a coding system and hence it will be easier to try the various methods of cluster analysis or other forms of categorical data analysis of which there are multitudes.

As an aside, a real problem you will have, whatever your approach, is that 300 participants is not really enough to explore a survey with 250 questions.

$\endgroup$
2
  • $\begingroup$ Thanks for your answer. After reading through your comments, I went to Amazon and purchased "Multiple Correspondence Analysis: 163 (Quantitative Applications in the Social Sciences)". Using that text, I was able to visualize the survey data appropriately. Thanks for steering me in the right direction! $\endgroup$
    – zbinsd
    Commented Jun 13, 2012 at 18:26
  • $\begingroup$ > 300 participants is not really enough to explore a survey with 250 questions . How many respondents are typically suggested for 300 questions? $\endgroup$
    – baxx
    Commented Oct 20, 2019 at 10:25
4
$\begingroup$

With the keyword "cluster" and "0/1 data", my knee-jerk reaction would be to put everything into a cluster analysis machine using a measure of "distance" between observations that only have binary variables. See e.g. Stata help file describing about a dozen such measures. I would run all possible analyses (hierarchical linkage/dendrogram) to see if there really are any interesting clusters. I would also get rid of the redundant answers that are perfectly collinear with one another (Cat == !Dog in your example); while they probably don't hurt in the simple analyses, they may bite in more complicated analysis and screw identification (Peter Ellis mentioned this in the comments).

More advanced, model-based, analysis can be performed along the lines of multidimensional item-response theory.

$\endgroup$
1
$\begingroup$

It turns out that I was greatly aided by the application of MCA (Multiple Correspondence Analysis). I have no affiliation with the author, but the book "Multiple Correspondence Analysis: 163 (Quantitative Applications in the Social Sciences)" helped vastly.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.