I want to statistically compare personality scores between different groups, but people can be in multiple groups. Is there anything problematic in just doing an ANOVA or a t-test since a respondent can belong to more than one group? Only problem I can see is that it's harder to get a significant difference, but that ist okay with the theory.
(similar question unanswered: Test like ANOVA but allowing for multiple group membership )
Below are two screenshot excerpts from "SPSS Algorithms" pdf document (you can easily find it on the internet), CTABLES procedure chapter. SPSS uses (in the Custom Tables procedure) a little-modified formula for the test statistic when the groups being compared are not disjoint but are "multiple response set" groups.
Here is the piece for testing between group means:
and here is between two proportions:
That was about pairwise comparisons between groups.
As for ANOVA, I expect it to be more problematic, because ANOVA is an omnibus test for k groups and thus puts more restrictive assumptions on the groups than a comparison of a pair of groups does. It will be harder to "correct for the groups' overlap". So don't do ANOVA, go directly to pairwise tests.
