It is usual to use all sorts of feature derivation you think are probably useful for your problem, then apply feature selection, feature reduction, and similar concepts to make best use of those features in your models. Not using a helpful feature derivation in the first place causes this information to be missing for your model, which will likely cause worse-than-possible results. So your feature selection and reduction approaches should account for not-helpful and redundant information (this is frequently coupled with training/evaluating models to avoid overfitting/selection bias).
Concerning robustness of the resulting model: this should rely on using a thorough evaluation approach and choosing a reasonable simple model from all evaluated models - instead of limiting feature derivation in the first place. Using repeated CV (training and evaluating different model types and parametrizations, then selecting best model based on resample performances) together with data partitioning appropriate for your setup (completely held-back test partition for estimating performance of final model, poss. done population independent like leave-subject-out-CV) will be OK for most setups.