1
$\begingroup$

I am doing weighted linear regression (W-OLS) with and without standardization (centering and scaling). The results are not identical and in many cases normalization is making the results worse. Can you please help me to figure out why this is happening? I suppose, normalization should not hurt. Is there any better way for normalization, e.g. subtracting min and dividing by (max-min)?

$\endgroup$
2
$\begingroup$

The coefficients should be different when you standardize (otherwise, why would you standardize?); data is standardized because you want to change the results. The t-statistics, p-values, and various goodness-of-fit measures should be unchanged. If they are changed, this means either:

  1. You are doing something wrong. That is, you are making a mistake of some kind.
  2. The independent variables have really large values (e.g., are in the thousands or greater), and you have numerical precision results. In this case, the understandardized values are incorrect.
$\endgroup$
  • $\begingroup$ If you're writing code yourself, make sure to convert back and that the data are unchanged. Don't forget that once you subtract the min, the max itself changes, so you simply divide by the new max (and not max - min) -- or just use MinMaxScalar. $\endgroup$ – photox Feb 20 '17 at 11:01
  • $\begingroup$ Tim, Thanks for your help. My concern is that when I do standardize I don't get generally better results. Some test data become better and some worse. This is something that I don't want to happen. And yes, the independent variables are very large, in order of 10^5. $\endgroup$ – user3720389 Feb 20 '17 at 18:01
  • $\begingroup$ Thanks Photox. I always convert back other wise the errors are so high and unreasonable. I add the mean of independent variable to it (mean (y), in case y = a*X) when I use z-score for standardization. $\endgroup$ – user3720389 Feb 20 '17 at 18:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.