Should stepwise regressions also be avoided for exploratory (hypothesis generating) modelling? In a recent paper,  Andrew Tredennick and colleagues (2021) suggested to use the drop1() function in R for exploratory modelling (that is to generate new hypotheses meant to be tested on new data). Their paper is very interesting but, since the drop1() function belongs to stepwise regression approaches (which are badly looked upon), I was wondering whether this procedure could truly be recommended even in this case?
My guess is that hypotheses generated that way will be shaky, at best. But sometimes, shaky hypotheses may perhaps be acceptable starting points to deepen your investigations on a phenomenon, especially when this phenomenon has seldom been investigated and/or data are extremely costly or difficult to collect.
EDIT: I have deleted the second part of my question because I asked it here.
EDIT 2 (Contextual clarification): Ultimately, my goal is to identify the most influencial variables controlling the biological phenomenon I'm studying. All the variables whose effects I want to explore are theoretically important to explain my response variable (they have already been selected based on my knowledge of the study system), but my sample size is simply too small to include them all as covariates in a single "inferential" model.
 A: Exploration means basically, that you can do whatever you want. If there are, among your too many variables, some which are actually true predictors you can hope for the drop functions to find them. You can then gather nuew data to infer, whether these are really actual predictors.
However, gathering new data comes with a cost (or you would have done that before). So the actual question is: Are you ready to gather more data based on nothing more then a stepwise regression approach?
That is not so much a mathematical/statistical question but depends on how costly gathering new data is and how much a positive result would be worth for your further research/career etc.
So, basically, see if you got any better options then stepwise. If not, perform stepwise to reduce the number of candidate predictors. If the result looks really promising, consider to sample new data on those predictors to do inferential statistics on.
A: In exploratory analysis you have much more latitude for how you generate the hypotheses of interest, since there is no biasing of tests due to optimisation processes.  (Of course, for this to apply you should ensure that when you undertake confirmatory analysis on the hypotheses, you use different data.)  Nevertheless, that does not mean that there is no difference in optimality of different kinds of processes that can be used to identify hypotheses of possible interest.  So while you can, in theory, "do whatever you want", you probably shouldn't.
Generally speaking, in exploratory analysis you will still want to identify hypotheses that have some evidentiary basis, so that you don't waste time testing lots of false hypotheses in the confirmatory phase.  For this reason, it is often useful to have regard to the same types of statistical/evidentiary issues that will arise in confirmatory testing, though for a different reason.  The main deficiency of stepwise methods in selection of variables is that it can travel through idiosyncratic paths that miss sets of explanatory variables with high evidence of a relationship to the response variable.  This is why comparisons like the all-possible-models method are considered preferable to stepwise methods.
Assuming you have sufficient computational power to do so, I would recommend you conduct exploratory analysis by computing the goodness-of-fit statistics for all possible models and then examining those models that should high levels of fit relative to the number of model parameters.  This method is more likely to identify models with true hypotheses, and unlike the stepwise procedure, it is more systematic and will not miss important models.  Since this is exploratory analysis, you should also allow yourself to be guided by exogenous concerns about what hypotheses/models are "interesting" in the context of your field, what are the costs of collecting data, etc., but you can use the all-possible-models method to augment this.  This latter method will give more systematic statistical information on your exploratory data than stepwise methods.
Finally, you say that hypotheses generated by the stepwise method are "shaky".  It is okay for hypotheses generated in the exploratory phase to be shaky, because the whole point is that you are only generating tentative hypotheses for later testing and confirmation.  Indeed, arguably all hypotheses generated in the exploratory phase are and ought to be "shaky".  The reason to prefer all-possible-models over stepwise methods is that it more systematically identifies hypotheses supported in the explanatory phase.
A: I think the assertion that stepwise selection is always unreliable is a bit too strong.
I think stepwise selection procedure can probably be amended to remove some of its weaknesses. One example might be to use a different sample from the same population in each step so that you are not inadvertently overfitting the model, and are only picking predictors or features that result in a model that generalises well enough for a purpose.
As with anything, when we are performing regression analysis, I do not think the notion that there is a "correct" model, or that there are "correct" or "incorrect" variables to choose is a useful one. I generally only care about a model's predictive performance, and its ability to describe the associations between the predictive and dependent variables. In fact, in many cases, the number of models that may produce an acceptable "fit" to the data and provide good predictive performance is probably very large, and it might be useful to think rather in terms of a set of models that are acceptable, and a set of models that are not, with the goal of a feature selection / model selection exercise being to identify a model that is in the set of acceptable ones.
But just my thought as a practitioner, rather than as a "scientist".
PS - I think the LASSO method, or other methods that use penalties and shrink coefficients, might well be slightly better founded, but I am not convinced they solve all of the issues with step-wise in any case.
