I have a dataset with ~600 datapoints, 49 categorical features (five possible categories), and a binary outcome variable. The dataset is incredibly imbalanced, with just over 3% of the outcomes in the positive class.
I've tried a number of algorithms using approaches such as traditional train/test split, SMOTE oversampling for the positive class, and optimism-corrected bootstrap as described Harrell and Efron/Gong.
The latter has given me by far the best results and I would like to apply some feature engineering by adding feature columns which represent the sums of all 49 features, as well as other columns which are the sums of certain subsets of the features.
My question is: can this feature engineering step be applied only on the original dataset from which the bootstrap samples are collected? Or does this step need to be part of the bootstrap loop (in other words, the feature engineering needs to be conducted every time a bootstrap sample is created)?
My instinct is that the feature engineering step is not required for every iteration in the bootstrap loop since the resulting column would be the same as from the original (un-bootstrapped) dataset.