1
$\begingroup$

I'm new to data science and Neural Networks in general. Looking around many people say it is better to normalize the data between doing anything with the NN. I understand how normalizing the input data can be useful.

However I really don't see how normalizing the output data can help.

I've also tried both cases with a easy dataset, and I achieved the same results. The only difference is that in some weird problems, it is really hard to then re-convert the output back.

Can you give me some intuition on why we should also normalize the output?

Or maybe why it is indifferent?

$\endgroup$
  • 1
    $\begingroup$ There is nothing magical in normalizing data. Think of it, say that you predict binary outputs: would it make any sense to re-scale the outputs to have mean 0 and sd 1 (= the output won't be binary)..? Say that you are predicting human age, would it make any sense for your predictions to have mean 0 and sd 1 ? Moreover, your algorithm gives you optimal predictions, if you arbitrary transform them, you make them suboptimal... $\endgroup$ – Tim Oct 31 '17 at 9:44
1
$\begingroup$

It can help because you have to update weights, and most optimization schema based on backpropagation through gradient descent impose a single step size for all weights in a layer.

If you have a single output node, then probably it won't help much. If you have more then it might make sense to normalize outputs. You can do it within the neural network architecture, then you get outputs in the correct scale anyways, but the neural network sees them normalized.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.