2
$\begingroup$

I am new to the field of deep learning, and I was wondering, whether there exist any theorems/laws, which govern how various preprocessing techniques effect the learning process.

I saw in some models, that values are beforehand scaled, or min-max-ed, yet I still do not have a clear picture as to what exactly works or does not.

Thank you very much.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

Preprocessing is quite a must to obtain a good result. It depends on the data type that you have and what you try to achieve. While for some cases like medical imaging, various techniques like special histogram equalization, intensity filtering are being used, in a NLP application, word2vec, vector normalization are being used.

All of their purpose are same:

  • Interpret data as meaningful features
  • Enhance features
  • Reduce noise
  • Normalize input

In the mathematical side, briefly, think that you have an image with pixel values ranging from 0-255. In that case, even a small change in the early layer weights / biases will have a huge impact on output, which we do not prefer to have. However, if you normalize the image between 0-1 or -1/1, weights can be distributed much more even across the layers and your network will have less overfit.

Improve this answer
$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.