# When and How to Scale Numeric Interaction Variables in Machine Learning Model

I found one similar question on cross validated, but it was unanswered; my apologies if this has been answered.

I'm experimenting with feature interaction in a regression model I'm working on in R. My only concern right now is with building and scaling/centering the features.

Is there a "more statistically correct" means of scaling/centering numeric interaction features? The options I've thought through include:

• option 1: Scale/center all numeric features, THEN calculate interaction values
• Secondary question here... do I need to then re-scale/re-center those values?
• option 2: Calculate interactions on all features, then scale/center it all at the same time

Well I had histograms to accompany the code below, but I don't have enough rep to post links to images.

# Example of option 1:

# taking a small sample of "airquality" data
set.seed(2)
my_aq <- data.frame(airquality[sample(1:nrow(airquality), 100), ])

# create a scaled/centered version
my_aq_pp_scaler <- caret::preProcess(my_aq, method=c("center", "scale"))
my_aq_scaled <- predict(my_aq_pp_scaler, my_aq)

# computing interactions with pre-scaled data
denmat_prescaled <- as.data.frame(model.matrix(~ .^2 - 1, data=my_aq_scaled))
hist(denmat_prescaled$Ozone:Solar.R, col='light blue', main="Pre-interaction-scale: Not Rescaled")  Then if I re-scale/re-center that, I'm left with this, which seems fine: # 1) do I need to scale/center again? denmat_pp_scaler <- caret::preProcess(denmat_prescaled, method=c("center", "scale")) denmat_prescaled_scaled <- predict(denmat_pp_scaler, denmat_prescaled) hist(denmat_prescaled_scaled$Ozone:Solar.R, col='light pink', main="Pre-interaction-scale: Also Rescaled")


I think this looks like what I would want from a machine-learning/modeling perspective. So if I go with option 1, I'd likely rescale/recenter.

# Example of option 2:

# postscaled - not scaling until AFTER interactions have been computed
denmat2 <- model.matrix(~ .^2 - 1, data=my_aq)
denmat2_pp_scaler <- caret::preProcess(denmat2, method=c("center", "scale"))
denmat_postscaled <- as.data.frame(predict(denmat2_pp_scaler, denmat2))
hist(denmat_postscaled\$Ozone:Solar.R, col='light green', main="No Pre-scale: Just Post-interaction-scale")


# Question:

Is one of these methods more statistically sound than the other? Or is this one of those "it depends" type situations. I find it interesting to see how much of an impact these different methods have on the overall skew of the final values as well. That was not something I anticipated. If anyone could apply a more statistically rigorous explanation of which is better and why it does/doesn't matter, that would be awesome. Thank you!

# edit:

The main model I'm using now is extreme gradient boosting (xgboost) with the objective set to "reg:linear" but I will likely also be trying lasso and ridge regression with glmnet.

# edit2

Due to some upvotes, I now have enough rep to add my histogram images.

• What type of regression are you doing with this data? Just glm, or other fancier models? – horaceT Oct 7 '17 at 17:43
• Sorry, I should have added that. The main model I'm exploring right now is extreme gradient boosting (xgboost) with my objective set to "reg:linear" but I will likely try several other strategies as well such as ridge and lasso regression with glmnet. – TaylorV Oct 7 '17 at 18:05
• GBM doesn't usually require input normalization. Your base/weak learner is a linear fit, it should be able to learn from raw. – horaceT Oct 7 '17 at 18:14
• Ah I see, that makes sense. But then it will matter more for when I start working with glmnet on ridge/lasso, right? – TaylorV Oct 7 '17 at 18:20
• Some algorithms require scaling inputs, such as neural net, but linear model doesn't care. I believe elastic net does. The only time you want to normalize inputs is when you want to compare coefficients. – horaceT Oct 7 '17 at 18:38