Does the $R^2$ (or some other statistic) from a univariate linear regression tell me anything about how it would work in a logistic model? What if I normalized the data to mean zero?

I'm doing multiple logistic regression, and I'm wondering if I created a ton of univariate linear regressions, if I can use the $R^2$ as an indicator of variables that are worth trying or not. Perhaps even adding a brute force search of interesting interactions or adding polynomials, etc.

  • $\begingroup$ Normalising the data should not change $r^2$ $\endgroup$ – Henry Mar 2 '14 at 20:50
  • 1
    $\begingroup$ this is the same as stepwise variable selection. see multiple posts on this topic elsewhere on this site $\endgroup$ – charles Mar 2 '14 at 21:59
  • $\begingroup$ why not doing it on the logit model directly? You'll get a better assessment that way.. $\endgroup$ – user603 Mar 2 '14 at 22:00
  • $\begingroup$ Thanks for your answers. The purpose of using Linear Regression was just to help with the computational complexity. ..I will read up on the backwards/forwards selection posts. $\endgroup$ – Jeremy Mar 2 '14 at 22:07

If you did want to do this (to foreshadow, you don't), it is not clear why you would want to use simple linear regressions. You would use simple logistic regressions instead. The response variable is distributed as a binomial, so linear regressions would be inappropriate (in addition to the whole procedure being inappropriate).

However, looking at each individual variable to see which is significant in isolation, and then adding those to a multiple logistic regression model is logically equivalent to a forward selection algorithm. Although this approach is very intuitive, it is unfortunately invalid (to understand this further, it may help you to read my answer here).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.