I implemented an artificial neural network using scikit neuralnetwork. As default configuration for my classification task I am using
10730 Datsets x 115 Features
1 Hidden Layer with 61 neurons
finding the minimum (~ 0.72) after 16-17 iterations
Now I tried to optimize the architecture in order to lower the mean categorical cross-entropy. At first I iterated the number of hidden neurons from 1 - 150, expecting the test-error to decrease until finding a minimum and then raising again.
(Note: For this plot I used the best/lowest error from all iterations for each of the 150 architectures. For another plot I used the mean error of all 50 iterations, which gave me the expected output (error decreases -> reaches minimum -> error rises), but I think it's the wrong calculation methodology, isn't it? )
So no matter how many hidden units I used, the error stayed on the same level. So I supposed, that the data is linearly separable and I don't need a Mult-Layer Perceptron.
I am confused whether the data is linearly separable or not. What am I missing? Why does the error not rise with additional hidden units, if it's not linearly separable?
Any help is welcome!