I am currently an undergraduate student beginning to explore the field of data science. Recently, our professor introduced us to the concept of standardizing a dataset.
My professor mentioned that normalizing/standardizing a dataset is performed in order to address the problem of features having different ranges/scales. He also mentioned that certain models are extremely sensitive to features having different scales (e.g. KNN) while other models simply don't care at all (e.g. decision trees).
My question today is, do any classification/regression models exist such that they actually perform worse on a standardized dataset than the original dataset itself?
The reason I ask this question is to decide whether or not to always normalize incoming data.
If the answer to my question is yes, then I will have to be smart about when to normalize, and when not to.
If the answer to my question is no, then I will just simply always normalize incoming data.