I know that MLP can separate classes even if the decision boundary is nonlinear. However, in all cases that I've seen the boundary was just a single 'unclosed' line.

  1. Is the MLP able to solve the problem when one class is inside the other? (i.e. the decision boundary is not unclosed line but a circle)

closed boundary

  1. What about the situation when one class is separated by some points of the other class? (i.e. two circles are needed)

two boundaries

  1. If the answer to any of the above is "yes", what is the minimal architecture that is able to do such separation (type of network, number of layers, number of units)?

1 Answer 1


Neural networks are universal function approximators–consequently, a single hidden layer with a tanh transfer function and many hidden neurons can separate the data, in theory.

That does not tell us anything about generalization though. It might be that the function separating the training data does horribly on unseen data.

If I had to do that ask, I'd use an MLP with elu transfer functions, 2 layers of 32 neurons each and residual blocks and maybe batch or weight normalisation. For optimisation, I'd use Adam. I suppose that should be able to solve it.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.