0
$\begingroup$

My model has 6 input features populated with continuous values (MinMax from -1 to 1) and 3 output.

The aim is to mutually identify one of three classes (multiclass single label).

I did tests for about a month trying out different configurations of the model using Mean Square Error as a cost function, getting some (not so exciting) results.

Then I read that for the Logistic Regression the MSE is absolutely wrong so I tried to use the (Softmax) Cross Entropy.

The problem is that using this function regardless of the model structure (layer number / number of neurons / activation functions) learning does not seem to work or at least the result is worse: the loss increases after a few epochs and accuracy is very low. What did i do wrong?

Old model (best configuration):

  • samples: 5100
  • batch size: 100
  • learning rate: 0.0001
  • loss function: MeanSquaredError
  • eval function: MeanAbsoluteError
  • 3 input
  • 1 hidden layer with 6 neurons, ativation: Tanh
  • 3 output, activation: linear
  • n. of epochs before loss increase: 8640
  • result: train loss=0,0040; eval loss=0,369

New model (best configuration):

  • samples: 5100
  • batch size: 100
  • learning rate: 0.01
  • loss function: CrossEntropyWithSoftmax
  • eval function: ClassificationError
  • 3 input
  • 1 hidden layer with 1 neurons, ativation: Tanh
  • 3 output, activation: linear
  • n. of epochs before loss increase: 2640
  • result: train loss=0,0049; eval loss=0,457
$\endgroup$
  • 4
    $\begingroup$ You can't compare loss numbers between models with different loss functions. It's apples and oranges. You need an objective metric, such as accuracy or AUC-ROC. $\endgroup$ – olooney Apr 3 at 13:29
  • 2
    $\begingroup$ Work better as judged using what? $\endgroup$ – Tim Apr 3 at 14:16
  • $\begingroup$ thank you, I learned a new thing, I made a wrong assumption $\endgroup$ – Rick Apr 3 at 15:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.