For my specific problem, but a common situation in the medical field, I have several hundred patients, and about 10-20 exaplnatory variables. the goal is to examine a specific predictor("treatment") for mortality.
considering a 20% event rate you'd get a high EPV, so it's desired to have a simpler model/less features to avoid overfitting or noise variables. It's rather pervasive to use univariate selection + stepwise for inference ("After adjustment, treatment A was significantly associated with reduced mortality"). From reading in CV, and also from Steyerberg/Harrell/ISLR and other online sources, this is generally a mistake, perhaps unless you have a very large sample size. The options I generally see, other than "stepwising":
- Using penalization/shrinkage (e.g. lasso) allows variable selection, but this does not readily translate for inference (in contrast to just getting coefs for prediction)
- Using propensity score for lumping covariates together (unless a risk score is available) - such as baseline characteristics.
- Using some sort of dimension reduction like PCA which I'm not familiar with
I can't select background characteristics based solely on literature (all might be relevant). Am I correct in my assessment? how do I make the best inference regarding the "treatment" variable?