Suppose we have a variable $Y_i$ which is a binary categorical variable, and we have a variables: $X_{i,1}$, $X_{i,2}$, ....$X_{i,n}$ ($i=1,2,....N$ where $N$ is sample size, and $n$ is the number of variables, and $n >N$), which are also binary categorical variables. And, suppose we want to understand the relationship between $Y$ and these variables individually (not controlling for a certain $X$). We could do $n$ Chi-Square tests (assuming the appropriate cell counts) between $Y$ and each $X_i$. However, with large $n$ (say >100) we would expect to have at least a few false positives. Using a Bonferoni correction might severely limit your power. Another alternative I was thinking of was a forward stepwise logistic regression where you only let a variable enter the model if the associated p-value is less than some criteria say 0.05. Yet, that has its own challenges like conveying statistical significance if the p-value of one variable changes by introducing another.

I suspect there is no "best practice", but please let me know if I am incorrect. My question is are any of these ideas just bad practice? Is there literature showing one of these ideas is much better than others? (I can't find any).

  • $\begingroup$ Depends on what you're trying to do. Are you trying to see if there is a bivariate association between $Y_i$ and each $X_n$, or do you know if there is a relationship between $Y_i$ and $X_n$, after controlling for other $X_n$? $\endgroup$ – Marquis de Carabas May 3 '16 at 21:59
  • $\begingroup$ Yes, not controlling for anything. I want to determine if there is a bivariate association between $Y$ and each $X_i$. $\endgroup$ – justin1.618 May 3 '16 at 23:21
  • $\begingroup$ What is your n? Please edit your post to include what the "very large" n is, as some users may not want to scroll through the comments for that detail. $\endgroup$ – Marquis de Carabas May 3 '16 at 23:24
  • $\begingroup$ Good point. I edited also to reflect that $n$ is not the sample size but the number of variables, and $n >$ the sample size ($N$). Though it would be interesting to discuss this at different levels of $n$ and $N$. $\endgroup$ – justin1.618 May 4 '16 at 13:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.