# Neural Network with improbable events

This is my first post, so let me know if I break any rules. I'm trying to learn about neural networks and I have implemented some toy examples; now I'm trying with a real data set.

My data set has about 70000 points $(x_i, y_i)$ where $x_i$ has about $30$ dimensions and $y_i$ is binary variable. I'm trying to train a feedforward multilayer NN with ReLu nonlinearities and softmax loss via stochastic gradient descent.

The problem I'm running into is that overall only ~1% of training cases has $y_i=1$, so my batches need to be pretty big to get any positive samples.

My question is generally what kind of things should I be paying attention to or tweak in order to deal with this kind of improbable event data.

Thanks!

• Artificially increase the probability of drawing data points having $y_i = 1$? For instance, see en.wikipedia.org/wiki/Importance_sampling . I'm just throwing this out there for your consideration, not saying ti is a good thing for you to do in this situation. – Mark L. Stone Jul 31 '16 at 0:22