0
$\begingroup$

I am building a regression model and I am trying to test whether my variables are correlated before using them in my model. However, I have some variables that are binary, i.e., they can assume the values of "true" or "false".

I have been told that I cannot use the Spearman rank to check the correlation of such variables, since I cannot say that "yes" is bigger than "no" or vice-versa. Someone suggested me to use the Jensen-Shannon divergence (JSD) to test the independence of such variables.

I noticed that JSD can compare two probability distributions and tell me whether they are similar or not. I am just confused about how am I going to represent my {true,false} variables as probability distributions? Also, is JSD really suitable for my situation? Or I should be better using another test such a Chi-Square test for independence? What are your suggestions?

Thanks!

$\endgroup$
  • $\begingroup$ Could you explain why a test of correlation is necessary? Lack of correlation is not one of the usual assumptions about any regression model. $\endgroup$ – whuber May 6 '18 at 19:24
  • $\begingroup$ Hi @whuber, because I want to be able to identify what variables share a strong relationship with my response variable. It will be harder to tell that if I have high-correlated variables, no? $\endgroup$ – Daniel Alencar May 6 '18 at 23:39
  • $\begingroup$ Yes, it will be harder. But there's nothing you can do about that unless you either change the model (to accommodate nonlinear relations--which is scarcely possible for binary variables), obtain additional data, or make stronger assumptions (such as Bayes prior distributions for the coefficients). Looking for a different test will be of no help. $\endgroup$ – whuber May 7 '18 at 11:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.