I'm working on a linear model with the "abalone" data set in R. I tried to build a more traditional linear model but found that the normality assumption was badly violated, even with a natural log transformation of the outcome. I instead decided to try bootstrapping. I ran 2,000 models, each time selecting features using stepAIC() and stepVIF(). I then analyzed which features "survived" within the 2,000 resulting models. Most of my predictors were present either ~100% of the time or ~0% of the time. One predictor however, was present ~28% of the time.
If a traditional linear model does not appear to work, is pivoting to a bootstrap technique a logical next step? Or at least not a ridiculous one?
Is this a feature selection process that makes sense? The other methodology I was thinking of was to run all of the models for all of the predictors, then only retain the ones that have confidence intervals entirely above or below 0. (as determined by quantiles of coefficients)
Assuming this is a valid feature selection process, what should I do about the predictor that shows up ~28% of the time. My instinct is to leave it out on the basis of a naive ~50% threshold, but somehow that feels off.
I also notice that this method gives one predictor labeled "Diameter" a negative coefficient related to the outcome ("Rings") even though they are positively correlated in the full data set. That seems like a red flag, right?
I know this is a long one and I appreciate the support.