Okay, this is a thought experiment: Suppose you have a dataset with 40 covariates. Suppose the data has every nice theoretical property you could want: randomly sampled, variables not correlated, etc. Suppose it is also the case that the null is true for each beta coefficient at a 95% level AND we don't know that the null is true in all cases. Next, you run the model, and 2 of the 40 betas appears to be significant, with a modestly large t value, but nothing crazy. Certainly enough to reject the null for both of these betas. Now, given that it was assumed the null is true, this is a perfectly reasonable result. Each beta has an approximately 5% chance of rejecting, so you would have **expected** two betas to reject before you even run the model! This brings me to my issue: in what sense can we say a beta is significant when we ALWAYS expect a certain number to reject just due to randomness? AND how is it possible to conduct hypothesis tests for these betas simultaneously? Isn't that cheating, sort of like rolling a die until you get a six? It seems to me you can only pre-specify a single beta to look at before you run the model to get "true" significance. Note that I am NOT asking about controlling the overall probability that the model is useful, in which case we can just do some CI adjustment with Tukey etc. PS: If you have an open-source text to cite and support your answer, I would love that.