I am working on bioinformatics data. I have a small dataset of around 60 rows. I have 3000 features. I need to build a regressor. I will be applying an initial round of feature selection to reduce the number of features to around 40. I am thinking of picking up the top 40 features which correlate well with the output variable (cross-validated feature selection the right way as suggested by https://www.nodalpoint.com/not-perform-feature-selection/ to avoid data leakage). Then I am planning to do an exhaustive feature selection of subsets of 3 or 4 features and input them to a Gaussian process regressor (to accomodate non-linear regressor). I will be doing feature selection and model evaluation using nested cross-validation. I hope I am taking all necessary precautions to build a model that can truly incorporate the relationship between my features and my response variable. However, because I have a huge initial feature set of 3000 features, I am afraid that I might end up picking some features that correlate well with my response variable just by luck or chance. And because I am doing exhaustive feature selection to select a smaller feature set, again chance might play a role.
What are the standard techniques to rule out the chance of luck in scenarios like this? Kindly share your inputs and links to any academic articles.
Many thanks for your time and attention!