I come across a problem where I trained two MLPs using the same dataset, but one was trained using the raw data and the second one was trained using the normalized version of the dataset. In this case, the MLP trained using the normalized dataset yielded better results. But I wanted to know, in general, will normalization always help in training an mlp? What possible dangers can normalization in this way pose to a supervised learning problem? are there dimensions that does not make sense to normalize for one reason or another? What are the implications if the normalization becomes a feature in the NN? What if the randomly initialized weights are a lot larger than the normalized numbers? I went through most of the threads that try to answer similar questions but tbh I didn't get a satisfying answer.

  • $\begingroup$ good question, I'm interested in this as well. Out of curiosity, were the variables in your raw data normally distributed? Why did you decide to use normalization in the first place, instead of, say, standardization, scaling or not transforming them at all? $\endgroup$ – johnjohn Feb 17 at 17:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.