A statistical estimator can be characterized – among others – by its bias and variance. These are different aspects, but if one is willing to accept a certain loss function (risk) than they can merged; the most popular example of which is MSE (which can be shown to be the square of the bias plus the variance). If one is willing to accept MSE as a metric, it can be used to objectively compare estimators even if they have different bias.
My question is: do we have analogous concepts for statistical tests?
Motivation: We all know that – despite being notoriously recommended, even by textbooks – it is a bad idea to select the applied test based on another test performed on the same sample. (Let's say for concreteness: to decide whether to use $t$-test or Welch-test based on an $F$-test, or Levene-test.) Doing so will result in an invalid test, i.e. the distribution of the $p$-values will no longer be uniform, the Type I error rate will be different from the significance level. However, one might say: "OK, I understand that there is some invalidity here, but, hey, we will have higher power!". (As $t$-test has higher power than Welch-test – that's of course the very reason why this flawed strategy is employed at all.)
Now, my feeling is that this invalidity in a statistical test is analogous to the bias of an estimator (actually, it is called bias in mathematical statistics texts; if I understand this concept correctly) and I have the feeling that power is analogous to variance.
So, question #1: are my feelings correct? Do we really have an equivalence here?
Question #2: if so, do we have an analog for MSE? (I accept that we have to select a loss function (risk) for this of course.)
This would be interesting because it would allow us to objectively decide whether there is any merit in the above reasoning (i.e. higher power offsetting the loss of validity – just as a biased estimator can have smaller MSE than an unbiased one).