I'm trying to implement a neural network in sklearn. I'm using stochastic gradient descent ('sgd') as the solver, an activation function of 'tanh' and all other values as the default ones provided by the library. I'm varying the value of hidden_layer_size from (10,1) to (100, 30) and noting the f1 score returned by 10- fold cross validation to find the optimal number of layers to keep in the model. However, my f1 scores continue to be constant at 0.8961, regardless of what the hidden layer size is.
Is this expected behavior?