If an exploratory factor analysis is done with some 1-5 agreement items and some 0/1 "choose all that apply" items, theoretically how much of a spurious tendency would there be for the 1-5 items to load on one or two factors and the 0/1 items to load on a separate set of one or two factors? (I've heard arguments both for and against the idea that correlations tend to be much higher among items of a similar scale. My own experimentation/simulation has not found much of an effect.)
- Anybody can ask a question
- Anybody can answer
- The best answers are voted up and rise to the top
The effect you mention happens because of response set, which can be controlled by the phrasing of the reactives, and the interest of the respondent, and the order of the questions. I've seen this happen in my own experience, but it's by no means inescapable: one good way to avoid the spurious tendency is to, say, put two agreement questions, then a dichotomy, then a couple more agreement ones, then the dichotomy, that usually does the trick for me.
A good way to test for that effect, if it's a scale you're building, is to compare the Cronbach's alpha of the entire scale with and without the dichotomies.
yes, factor loadings might be affected by the range of possible responses, but it won't neccesarily be dominated by it, if you take the measures you would normally take to avoid response set.