# How to improve neural network sensitivity with a lopsided binary outcome?

I'm working on predicting (not explaining) a 0/1 outcome that generally has only about 10% "1"s (I'm not at liberty to name the variables). N ~40,000. Logistic regression proved unsatisfactory, both when using about 5-10 main effects and after I built in several interaction terms suggested by CHAID procedures. Sensitivity was ultimately only about 25%.

I then turned to neural networks (radial basis function networks, in SPSS). I was pretty shocked to see the program fail to classify any cases as "1"s. That is, sensitivity was zero. First question: Is this a common or understandable NN result under these conditions?

Next I tried randomly excluding a large number of "0" cases in the training set, bringing the fraction of "1"s up to about 40%. Now the program was able to correctly identify a halfway-decent number of cases in the training set, with sensitivity around 30%, but that dropped to 20% when the solution was applied to the test set, which once again consisted of only about 10% "1"s.

2nd question: How would you get around this problem?

-

Suppose bad students pass a tough course with probability $0$, while good students pass the course with probability $1/3$. If the only information you get to observe is whether the student is good or bad, then your most accurate prediction is that the student will fail every time. You may learn from the training data that a good student is more likely to pass than a bad student, but you will never believe that a particular student is more likely to pass than to fail.
If you are trying to fill in missing target values (to generate data) then instead of filling in the most likely, you could fill in a random value whose probability distribution is given by your model. If your model says that target value A happens $35\%$ of the time, then you fill in A $35\%$ of the time. In some situations that is much better than filling in the most likely value $100\%$ of the time. E might be the most common letter in English but not many sentences consist of only Es. –  Douglas Zare Dec 24 '12 at 16:52