I'm currently testing some feature selection methodologies/algorithms in R, like the Recursive Feature Elimination from the R caret package, and also the RRF R package, to select a subset of features from a merged dataset, which is comprised by both gene expression data, and some continuous clinical variables. My main goal, is to inspect via any of these methodologies, in any(or most) of the clinical variables are selected along with the gene features, in the "subset" returned from the feature selection procedure, in order then to test this "composite" set in downstream procedures. My main question(although might sound very naive), is about the random seed issue and the reproducibility of my results. In detail, both the rfe function from caret, as also the RRF function from the RRF package, need before each run, a random seed number in order for the results to be reproducible. However, how i can ensure via a more "appropriate" methodology, that the subset of features selected is similar, and not just dependent on the specific random seed ?
In other words, should i test with either methodology, a set of different random seed numbers, through the argument:
and then inspect which features are commonly selected ? And implement this somehow with a for loop ?
I insist on this matter, because i would like to perform various statistical methodologies with this selected subset, and it would be preferable to be as much as less "random".