I found one similar question on cross validated, but it was unanswered; my apologies if this has been answered.
I'm experimenting with feature interaction in a regression model I'm working on in R. My only concern right now is with building and scaling/centering the features.
Is there a "more statistically correct" means of scaling/centering numeric interaction features? The options I've thought through include:
- option 1: Scale/center all numeric features, THEN calculate interaction values
- Secondary question here... do I need to then re-scale/re-center those values?
- option 2: Calculate interactions on all features, then scale/center it all at the same time
Well I had histograms to accompany the code below, but I don't have enough rep to post links to images.
Example of option 1:
# taking a small sample of "airquality" data set.seed(2) my_aq <- data.frame(airquality[sample(1:nrow(airquality), 100), ]) # create a scaled/centered version my_aq_pp_scaler <- caret::preProcess(my_aq, method=c("center", "scale")) my_aq_scaled <- predict(my_aq_pp_scaler, my_aq) # computing interactions with pre-scaled data denmat_prescaled <- as.data.frame(model.matrix(~ .^2 - 1, data=my_aq_scaled)) hist(denmat_prescaled$`Ozone:Solar.R`, col='light blue', main="Pre-interaction-scale: Not Rescaled")
Then if I re-scale/re-center that, I'm left with this, which seems fine:
# 1) do I need to scale/center again? denmat_pp_scaler <- caret::preProcess(denmat_prescaled, method=c("center", "scale")) denmat_prescaled_scaled <- predict(denmat_pp_scaler, denmat_prescaled) hist(denmat_prescaled_scaled$`Ozone:Solar.R`, col='light pink', main="Pre-interaction-scale: Also Rescaled")
I think this looks like what I would want from a machine-learning/modeling perspective. So if I go with option 1, I'd likely rescale/recenter.
Example of option 2:
# postscaled - not scaling until AFTER interactions have been computed denmat2 <- model.matrix(~ .^2 - 1, data=my_aq) denmat2_pp_scaler <- caret::preProcess(denmat2, method=c("center", "scale")) denmat_postscaled <- as.data.frame(predict(denmat2_pp_scaler, denmat2)) hist(denmat_postscaled$`Ozone:Solar.R`, col='light green', main="No Pre-scale: Just Post-interaction-scale")
Is one of these methods more statistically sound than the other? Or is this one of those "it depends" type situations. I find it interesting to see how much of an impact these different methods have on the overall skew of the final values as well. That was not something I anticipated. If anyone could apply a more statistically rigorous explanation of which is better and why it does/doesn't matter, that would be awesome. Thank you!
The main model I'm using now is extreme gradient boosting (xgboost) with the objective set to "reg:linear" but I will likely also be trying lasso and ridge regression with glmnet.
Due to some upvotes, I now have enough rep to add my histogram images.