Okay, this is a thought experiment:

Suppose you have a dataset with 40 covariates. Suppose the data has every nice theoretical property you could want: randomly sampled, variables not correlated, etc. Suppose it is also the case that the null is true for each beta coefficient at a 95% level AND we don't know that the null is true in all cases.

Next, you run the model, and 2 of the 40 betas appears to be significant, with a modestly large t value, but nothing crazy. Certainly enough to reject the null for both of these betas. Now, given that it was assumed the null is true, this is a perfectly reasonable result. Each beta has an approximately 5% chance of rejecting, so you would have expected two betas to reject before you even run the model!

This brings me to my issue: in what sense can we say multiple betas are significant when we ALWAYS expect a certain number to reject just due to randomness? AND how is it possible to conduct hypothesis tests for these betas simultaneously? Isn't that cheating, sort of like rolling a die until you get a six? It seems to me you can only pre-specify a single beta to look at before you run the model to get "true" significance.

To clarify what I mean by simultaneous: I am not talking about the overall test (F-Test, for example) or any related mitigating actions like multiple comparison corrections. I fully agree that these tests and measures are effective and make sense. I am asking at the level of the individual betas themselves with their individual CIs: if two betas reject, should I just ignore this and say: "Well, I expected two to reject. These effects are not real"? This may be my main point: does significance at the regression coefficient level mean anything in this case? Or suppose four had rejected. I would guess two would be false signals by chance, but how do I know which is real and which is fake?

PS: If you have an open-source text to cite and support your answer, I would love that.