I have a feed forward ANN with three outputs. The order of magnitude of two of the outputs are the same (in the range 10^1). However the third output's magnitude is about two orders of magnitude less than than the first two inputs. I am able to train the network so as to get very low error on the first two outputs. But no amount of training seems to make reduce the error for output three. Is this a scaling problem or is it because my network is not large enough.
Network details. My input layer has 5 inputs and two hidden layers with 4 neurons. Activation function is ReLU. I am standardizing the input but not the output.