# Logistic regression and clustering?

This is regarding a binary classification with logistic regression with one independent variable (IV) and one dependent variable (DV). The IV seems to be distributed normally, the residuals have a random distribution, and AS FAR AS I KNOW, based on my domain knowledge and looking at a plot, it should affect the DV in a linear manner, and it is definitely a monotonic relationship. So basically, it should be a very straight forward relationship. There are 1100 datapoints.

When I run the regression and check the predictions with 10fold cross validation, it predicts at about 51% accuracy or even just randomly. There is a lot of variance so the best we can hope for is maybe 54%. It is worth noting the model predicts a success at about 4 to 1 rate when the true rate is close to 1 to 1.

Question 1: At this point, if you were trying to improve this regression, what would you try to do? What do you think may be the problem?

So I tried various things like adding different IV's, trying random transformations, and had no success. Then I started clustering the data and only selected ABOUT half the datapoints which most closely match the test case I am trying to predict. This improved the accuracy greatly.

Question 2: Why would clustering help in such a simple, basic, regression situation? Is there a more philosophically sound, more effective way than clustering to set up this regression?

• I'm not quite sure you mean when you say "The best we can hope for is maybe 54%". Assuming you're referring to correctly predicting the class labels and that the outcomes aren't nearly 50/50 coin flips, you can probably do better than that without a model at all. For example, if the proportion of $1$s in the data set is $p>1/2$ (if $p<1/2$ just flip the labels). Then you will get a prediction accuracy of $p$ just by guessing everything is a $1$. Maybe I've missed your point.. – Macro Jul 25 '13 at 17:15
• Marco, in this situation p is designed to equal 50%. I try to find "hidden information" to improve accuracy prediction above 50% – appleLover Jul 25 '13 at 22:17