I had an idea, and was wondering if it's ever done and, if so, how to do it in an appropriate manner.
Let's say I run an OLS model and the results come back significant. There is discussion as to whether the positive results hold up given a different set of control variables. However, it's theoretically unclear which control variables are actually relevant, and we don't just want to throw in the kitchen sink.
Let's say it's computationally feasible to run every single model (with every possible permutation of control variables), and we collect the coefficient and standard error of the coefficient of interest in each. We can report the proportion of models where this coefficient is significant (correcting for multiple tests). Is there some kind of omnibus procedure that can be done showing that the test statistic is significant in a significant number of alternative specifications? Is this ever done?