How to use real data for neural nets without clear input boundaries

Question

I'm trying to use a neural net for classification and I am getting stumped on how to use it for noisy data. My problem seems simple; I have 1000's of experiments that have features and these features either hit a threshold or don't. I know if the experiments are success' or not, but since the data is noisy I can have two inputs map to different outputs, e.g.,

[1,1,0,0,0,1] -> 1
[1,1,0,0,0,1] -> 0

What I don't understand is how to use neural nets when I have thousands of runs to classify. Most of the examples online deal with XOR functions where there is a 1-1 mapping between input and output like here. How can I use neural nets to classify these inputs where the maps are not 1-1?

If I wanted to use the code in the link how can I do it? The weights would correspond to the same input, no? For example, how would I construct a neural net that could be written as this:

   nn = NeuralNetwork([6,2,1])
    X = np.array([[1,1,0,0,0,1],
                  [1,1,0,0,0,1],
                  [1,1,0,0,0,1],
                  [1,1,0,0,0,1],
                  [1,1,0,0,0,1],
                  ...])
    y = np.array([1, 0, 0, 1, 0,...])
    nn.fit(X, y)
    for e in X:
        print(e,nn.predict(e))

where the ... means more data and of course, I have other inputs and outputs, [1,1,1,1,1,1] -> 1, etc.

Answer 1

You simply feed all input-output pairs into the training algorithm. There is no requirement in neural networks that the training data exhibit deterministic separability as a function of the input values. Most real-world problems will involve training data that are not perfectly separable, which is expected when there is some noise in the responsible generating process.

Most useful classifiers will respond to this condition by reflecting the tendency in a given decision region to be a 1 or a 0 (in a binary classifier). This tends to manifest as greater uncertainty (equivalent to steeper gradient of pre-thresholded classifier output value with respect to the inputs) in regions where greater overlap is seen in the training data. This will be true of most classifiers, including neural networks.

A true probabilistic classifier may express this explicitly in terms of probability. A neural network won't necessarily do this as it depends on structure, although if a logit activation is used for the output layer (just for instance) it will share some characteristics of probabilistic logistic regression. However, one has to be careful with any strong statements about equivalency here because the function of preceding layers will influence outcomes greatly depending on choice of activations, layer depth, number of hidden features, and so forth.

As an aside: your sample data shows no variation in the input feature space (but the outputs vary). That's not likely to result in anything useful in terms of training since these data are purely co-linear (stochastic gradient descent needs to be able to sample many different points in the input space to produce anything other than trivial behavior from the network).