My model has 6 input features populated with continuous values (MinMax from -1 to 1) and 3 output.
The aim is to mutually identify one of three classes (multiclass single label).
I did tests for about a month trying out different configurations of the model using Mean Square Error as a cost function, getting some (not so exciting) results.
Then I read that for the Logistic Regression the MSE is absolutely wrong so I tried to use the (Softmax) Cross Entropy.
The problem is that using this function regardless of the model structure (layer number / number of neurons / activation functions) learning does not seem to work or at least the result is worse: the loss increases after a few epochs and accuracy is very low. What did i do wrong?
Old model (best configuration):
- samples: 5100
- batch size: 100
- learning rate: 0.0001
- loss function: MeanSquaredError
- eval function: MeanAbsoluteError
- 3 input
- 1 hidden layer with 6 neurons, ativation: Tanh
- 3 output, activation: linear
- n. of epochs before loss increase: 8640
- result: train loss=0,0040; eval loss=0,369
New model (best configuration):
- samples: 5100
- batch size: 100
- learning rate: 0.01
- loss function: CrossEntropyWithSoftmax
- eval function: ClassificationError
- 3 input
- 1 hidden layer with 1 neurons, ativation: Tanh
- 3 output, activation: linear
- n. of epochs before loss increase: 2640
- result: train loss=0,0049; eval loss=0,457