I have often heard people mention the importance of standardization when using PCA, but not with respect to GAMs. Is this because standardization is not important for GAMs? Should I standardize my predictors and/or response variables?


(I am intepretting your use of "PCA" to mean "principal components regression"). It is important to standardize variables in PCA so that you can compare model coefficients as being on the same scale. For example, if PC1 has a coefficient of 0.9, and PC2 has a coefficient of 0.01, you have some context for their relative importance. Scaling is not important if you do not care about coefficients - your predictions will be the same.

To answer this question for GAMs, it greatly depends on what kind of GAM you are fitting (there are many). In any case, scaling can play an important role in regularizing your model. Take, for example, the LASSO (which is a very, very reductive case of a GAM). It minimizes:

$$\|\mathbf{y - X\beta}\|^2 - \lambda\|\beta\|_1$$

Without scaling your data, that coefficient vector $\beta$ will have values all over the place. Larger elements of $\beta$ will be penalized more than smaller elements - even if the smaller element was less important to prediction.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.