1
$\begingroup$

I have a feed forward ANN with three outputs. The order of magnitude of two of the outputs are the same (in the range 10^1). However the third output's magnitude is about two orders of magnitude less than than the first two inputs. I am able to train the network so as to get very low error on the first two outputs. But no amount of training seems to make reduce the error for output three. Is this a scaling problem or is it because my network is not large enough.

Network details. My input layer has 5 inputs and two hidden layers with 4 neurons. Activation function is ReLU. I am standardizing the input but not the output.

$\endgroup$
1
$\begingroup$

It's most likely a scaling issue: make sure that the cost function reflects the importance you give to each of the three outputs.

$\endgroup$
  • $\begingroup$ In the cost function I did use multipliers to increase importance the 3rd output. $\endgroup$ – siby Sep 12 '16 at 17:52
  • 2
    $\begingroup$ @siby one safety check would be to ignore the other two outputs and see how well the third output can be predicted in isolation. $\endgroup$ – Franck Dernoncourt Sep 12 '16 at 17:53
  • $\begingroup$ @Farnck I checked it with cost function only for 3rd output error. The prediction is not good. However when I checked it only for 1st and 2nd output error the prediction was as good as when I had 3 outputs. $\endgroup$ – siby Sep 12 '16 at 19:33
  • 1
    $\begingroup$ @siby Then it might simply be impossible to accurately predict the third output. You could try to predict your third output with some basic classifier e.g. SVM to confirm. $\endgroup$ – Franck Dernoncourt Sep 13 '16 at 1:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.