I have responses from 3000 respondents to a number of survey questions and their options. Treating them as binary variables, I get a matrix of respondents vs 1/0 responses (like eg Surveymonkey expanded XLS format). I am trying to see if there are correlations among them of interest.
Start with a simple total of all responses so I know what % each response is of the overall population.
TL;DR: simple significance testing seems to show me pairs of responses that are correlated but other methods give very low results.
First method is just to get the responses to other options of all who have picked one option. That's a simple table, and is easy to understand. But gives no sense of possible multiple options in common.
I do Pearson correlations but all are less than 0.2 and 0.1 is common. Pretty low. It looks insignificant, but when I tried 3:
I do a z-test to see if the % of any subgroup for a given other option is significantly different from the population and many are significant which is interesting and the combinations also intuitively look like they belong together.
A chart of Z test vs correlation is so much like a straight line that I suspect I am really looking at some similar mathematical method at root - it's been so many years since I studied statistics that I can't see what that is.
- I do a Jaccard index and while there seems to be a vague connection it's all over the place so once again I wonder am I trying to see relationships that are not there.