Normalizing/Scaling a dataset does not have any effect on r2 score? I have this question on my mind for some time now, but unable to find some thorough explanation around this. While working on the Boston housing data set, scaling the data has no effect on the output. In my understanding, it should have increased the r2 score as some weights would have scaled-down and their effect would have been minimized after scaling. Is my interpretation of this concept wrong. How do I explain this to someone whom I just told that it would help improve the results? Does it works only sometimes, does it have any performance effect?
 A: I'm assuming that by "normalizing" you probably mean either 1) subtracting the mean and dividing by the standard deviation for that variable (often called standardizing) or 2) subtracting the minimum and dividing by the range (maximum - minimum) to produce values between 0 and 1. Linear transformations like this, when consistently applied to all of the data for that variable, can only change the units of that variable and not its meaning.
If you have a temperature in degrees Fahrenheit and you want to convert it to degrees Celsius, you can subtract 32 and divide by 9/5, and so 50 °F and 10 °C have the same meaning. If you standardize a variable by subtracting the mean and dividing by the standard deviation, while the latter might have been obtained from the data, it's the same process. The data is then in units of standard deviation relative to the mean.
If you have the distance a heavy object can be thrown measured in centimeters, you can convert these to inches by dividing by 2.54. In this case, 25.4 cm is the same as 10 inches.
These sorts of transformations, including dividing by the standard deviation of the data at hand, because this is the same value being used for all observations, won't change the information in the temperature or distance data, and so the proportion of variance one of these explains in the other won't change either.
The usual reasons for standardizing or normalising are for interpretation or numerical stability. These don't change the actual meaning of the model and so don't improve its fit (assuming the model is estimated without numerical issues).
