Can a neural network classify a new input in the same coordinate of another but have different output label?

Question

In a 3D model, say we have the following input:

x---y---z---Label

0---0---0---A

1---0---0---A

1---1---0---A

0---1---1---A

1---0---1---B

1---1---1---B

Now, there's a new input that we want to add to the previous data with the (0,1,1) coordinates. If the new input is labeled "A", a NN can easily separate perfectly the data in two classes (a simple regression would do it).

However, if it is labeled "B", we will have two data that have the same coordinates but different labels. How does a NN with, say one hidden layer, deal with this kind of problem?

My guess is that the problem becomes not linearly separable (NN are good with that) however, they are at the same spot so we would not be able to separate perfectly the data between A and B. Am I correct?

Answer 1

The actual problem is not just separability, but conflicting information. When you have it, not only neural nets but any deterministic decision mechanism will produce wrong result for one of the samples. There is no way to deal with it. And, depending on your data or method, sometimes your training example might be classified wrongly while the test sample is being classified correctly.