I have a number of dichotomous variables from a number of study participants, and I want to do some exploratory factor analyses and other things. However, my sample is too small to include all the dichotomous variables, so I have to select a subset of them. There are various ways in which they can be selected, such as by theoretical considerations, agreement with another particular variable which measures an underlying construct of interest etc.

But I also have an idea that variables with a more even distribution of 1:s and 0:s should be preferable to variables with a very uneven distribution. My thinking is that variables in which only 1% are 1:s, or 99% are 1:s, contribute with very little information regarding any underlying factors. It thus seems intuitive that the closer the distribution of the variable is to 50% of each, the more information the variable potentially may contain regarding any underlying factor structure. So it would then make sense to try an approach in which this relationship is maximized, i.e. selecting variables based on closeness to a 50/50 distribution.

Is my thinking about this issue correct - is this a viable approach? And if so, could anyone provide a reference in which this issue is discussed, so that I can reference it if I need to?

  • $\begingroup$ What is the objective of this analysis? If ultimately you are looking for variables that might be associated with some kind of outcome, there seems to be little basis to exclude variables with highly uneven distributions. Wouldn't those just as easily be the red flags that clearly indicate the presence or absence of the outcome? $\endgroup$
    – whuber
    Commented Mar 18, 2021 at 18:54
  • $\begingroup$ I agree; this seems to be the same fallacy as in this question: that marginal entropies alone can lead to choices based on conditional entropies. $\endgroup$ Commented Mar 25, 2021 at 12:41
  • $\begingroup$ See also this question, whose answer provides a way to identify redundant variables. $\endgroup$ Commented Mar 25, 2021 at 12:45
  • $\begingroup$ Thank you for the comments, both of you. I'll look into the links provided by @AryaMcCarthy, and if I'm not satisfied, I'll edit my question. $\endgroup$
    – JonB
    Commented Mar 26, 2021 at 7:24


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.