What does statistical power mean when we are interested in the probability of correctly not rejecting the null hypothesis? It seems that statisticians are usually only interested in statistical power. In other words, they are interested in the probability of correctly rejecting the null hypothesis. 


*

*What if we are interested in the probability of correctly not rejecting the null hypothesis?

*Would we change the hypothesis test so that we are computing a power?

 A: The probability of "correctly not rejecting the null hypothesis"--i.e., if the null hypothesis is true, we do not reject it--is controlled by the significance level at which we are doing the test. If I choose a significance level of $\alpha = .05$, so that I reject if my $p$-value is less than .05, then my probability of correctly not rejecting the null hypothesis is $1-.05 = .95$. If rejecting the null hypothesis when it is in fact true would have very bad consequences then we might use a smaller $\alpha$, say .01 or even .001, which gives us a higher probability of correctly not rejecting the null hypothesis.
So, in fact, we already control this probability--in fact, this is much easier to control than the power. Because it's much easier there's much less discussion about it, which is probably why you concluded that statisticians aren't interested in it.
A: I think you are interested in equivalence testing. See this other question on testing a hypothesis of no group differences. 
There are various approaches that can be adopted to assess whether the null hypothesis is true. In general, the absence of statistically significant effect is very week evidence for the truth of the null hypothesis.
Three common approaches include (a) looking at confidence intervals; (b) looking at bayesian posterior densities on the parameter of interest; or (c) setting up two one-sided significance tests.
The confidence interval and Bayesian posterior density approach are often used to quantify uncertainty of a parameter of interest. The Bayesian approach is arguably more aligned with the question of interest where the parameter is seen as unknown. Looking at such intervals you could judge that if the interval includes the null hypothesis and other plausible values are sufficiently close to zero, then this means that the null hypothesis or something sufficiently similar is the most likely the truth.
A similar approach is to set up two one-sided significance tests. E.g., when testing whether the means are the same for two groups ($\delta = \frac{\mu_1 - \mu_2}{\sigma}$) you could test whether $\delta$ is significantly less than .1 and significantly more than -.1.
In this cases you could calculate the statistical power of such tests assuming:


*

*the null hypothesis is true

*alpha

*sample size 


Or if you wanted to hold power constant, then you could assess what sample size would be required. You could also vary the threshold for equivalence and see how as you expand the width of the equivalence threshold your power increases.
This is a common applied problem in the context of equivalence and non-inferiority testing for drugs (e.g., Walker and Nowacki, 2011).
References


*

*Walker, E., & Nowacki, A. S. (2011). Understanding equivalence and noninferiority testing. Journal of general internal medicine, 26(2), 192. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3019319/
