Do you ever center AND standardize variables in multiple regression? It seems as if standardization would automatically center variables...is this true?
1$\begingroup$ See also, e.g., for if and how stadardization may or may not matter: stats.stackexchange.com/questions/348758/… stats.stackexchange.com/questions/188715/… stats.stackexchange.com/questions/208341/… $\endgroup$– Christoph HanckJun 14, 2018 at 9:51
Yes, standardizing usually implies centering, so if you're standardizing, you are necessarily centering in the process. Centering is sometimes done without standardizing, e.g., when the original metric of variables is worth preserving, but when one wishes to remove nonessential multicollinearity*. Here are a few sources to consult for more info:
- Wikipedia's "standard score" page
- A page from UCLA's Stat Consulting Group's Stata FAQ
- A recent answer with an interesting thread of comments about the ambiguity of "standardize"
- * Dalal, D. K., & Zickar, M. J. (2012). Some common myths about centering predictor variables in moderated multiple regression and polynomial regression. Organizational Research Methods, 15(3), 339–362. Retrieved from https://umdrive.memphis.edu/dsherrll/public/SCMS8540/Dalal%20%26%20Zickar-2012.pdf.
A linear transformation on the inputs does not affect the predictive power of a multilinear model, think of it as converting from Celsius to Fahrenheit. Your model stays the same, although the coefficient attributed to the variable will naturally reflect the normalization.
If you are using the norm of the model coefficient to estimate feature importance, you should pay attention to properly normalize your variables and watch for correlations in order to effectively use weight norms as proxy for feature importance.
$\begingroup$ This nicely answers one aspect of the question. Another aspect concerns numerical stability in the solution. One purpose of standardizing the independent variables is to reduce propagation of rounding error in computation of the solution. The issue is that as a practical matter--for computation with finite precision arithmetic--linear transformations can affect the predictive power of the model! $\endgroup$– whuber ♦Jun 14, 2018 at 12:27