Null hypothesis of equivalence

Question

Suppose $X_1, X_2, \, ... \, , X_n$ are a simple random sample from a Normal$(\mu,\sigma^2)$ distribution.

I'm interested in doing the following hypothesis test: $$ H_0: | \mu| \le c \\ H_1: |\mu| > c, $$ for a given constant $c > 0$.

I was thinking of performing two one-sided $t$-tests (TOST) in an analogous way to the usual bioequivalence testing situation, where the null and is $|\mu| \ge c$ instead, but I don't know if this makes sense or is correct.

My idea is to perform the one-sided tests $$ H_{01} : \mu \le c \\ H_{11} : \mu > c $$ and $$ H_{02} : \mu \ge -c \\ H_{12} : \mu < -c, $$ and reject the global null hypothesis if one of the $p$-values is smaller than a significance level $\alpha$.

Thanks in advance!

EDIT:

I've been thinking a little while about this, and I think the approach I proposed does not have significance level $\alpha$.

Suppose that the true value of $\mu$ is $\mu_0$ and $\sigma^2$ is known.

The probability of rejecting the null in the first test is $$ \mathbb{P}_{\mu_0}(\mathrm{Rej.H}_{01}) = 1 - \Phi \left(z_{1-\alpha} + \frac{c-\mu_0}{\sigma/\sqrt{n}} \right), $$ where $\Phi$ if the standard cdf of the Normal distribution, and $z_{1-\alpha}$ is a value such that $\Phi(z_{1-\alpha}) = 1-\alpha$.

If $\mu_0 = c$, $\mathbb{P}_{\mu_0}(\mathrm{Rej.H}_{01}) = \alpha$. Then, if $\mu_0 > c$, $\mathbb{P}_{\mu_0}(\mathrm{Rej.H}_{01}) > \alpha$. Alternatively, if $\mu_0 < c$, $\mathbb{P}_{\mu_0}(\mathrm{Rej.H}_{01}) <\alpha$.

The probability of rejecting the null in the second test is $$ \mathbb{P}_{\mu_0}(\mathrm{Rej.H}_{02}) = \Phi \left(- z_{1-\alpha} - \frac{\mu_0 + c}{\sigma/\sqrt{n}} \right). $$

Again, if $\mu_0 = -c$ we have $\mathbb{P}_{\mu_0}(\mathrm{Rej.H}_{02}) = \alpha$. Similarly, if $\mu_0 > -c$, $\mathbb{P}_{\mu_0}(\mathrm{Rej.H}_{02}) < \alpha$. Finally, if $\mu_0 < -c$, $\mathbb{P}_{\mu_0}(\mathrm{Rej.H}_{02}) > \alpha$.

Since the rejection regions of the two tests are disjoint, the probability of rejecting $H_0$ is: $$ \mathbb{P}_{\mu_0}(\mathrm{Rej.H}_0) = 1 - \Phi \left(z_{1-\alpha} + \frac{c-\mu_0}{\sigma/\sqrt{n}} \right) + \Phi \left(- z_{1-\alpha} - \frac{\mu_0 + c}{\sigma/\sqrt{n}} \right) $$

So, if $\mu \in [-c,c]$, $2\alpha$ is an upper bound of the probability of rejecting the (global) null hypothesis. Therefore, the approach I proposed was too liberal.

If I'm not wrong, we can achieve a significance level of $\alpha$ by doing the same two tests and rejecting the null if the $p$-value of one of them is less than $\alpha/2$. A similar argument holds when the variance is unknown and we need to apply the $t$-test.

Answer 1

Very interesting question!!

You are using the logical consequence, i.e., the entailment condition. This entailment condition forms the very basis of the classical logic, it guarantees the inference or deduction of a result from a premise.

The reasoning behind your proposal follows:

If $H_0$ entails $H_0'$, then the observed data should draw more evidence against $H_0$ than $H_0'$.

In terms of your auxiliary hypotheses $H_{01}$ and $H_{02}$, we have $H_{0} \equiv H_{01} \wedge H_{02}$, that is, $H_{0}$ entails $H_{01}$ and also $H_{0}$ entails $H_{02}$. Therefore, according to the entailment condition, we should observe more evidence against $H_{0}$ than either $H_{01}$ or $H_{02}$. Then, you concluded that if one of the p-values computed under $H_{01}$ or $H_{02}$ is sufficiently small, the p-value computed under $H_0$ will be even smaller.

However, this logical reasoning is not valid for p-values, i.e., p-values do not respect the logical consequence. Each p-value is build under a specific null hypothesis, therefore, p-values for different null hypotheses are computed under different metrics. For this reason p-values cannot respect the logical reasoning over the parameter space (or the space of the null hypotheses).

Examples where p-values violate the entailment condition are presented in Schervish (1996) and Patriota (2013). The latter paper shows examples from a bivariate normal distribution and from a regression model (see Examples 1.1 and 1.2 on pages 5 and 6, respectively). Eran Raviv provides an algorithm in R code for the bivariate case. The learning from these examples is: you must compute the p-value directly for the null hypothesis of interest. Schervish (1996) provides a p-value's formula for your example when $n=1$ and $\sigma^2 = 1$, see Formula (2) on page 204. If you want to compute a p-value, you must adequate that formula for your case.

Patriota (2013) proposes a new measure of evidence to test general null hypotheses (composite or simple null hypotheses) that respects the logical consequence. This measure is called s-value in the paper. The procedure is relatively simple for your example:

1. Find a (1-$\alpha$) confidence interval for $\mu$ (an asymptotic one): $I(\mu, \alpha) = \bigg[\bar{x} - z_{\alpha/2} \sqrt{\frac{s^2}{n}} \ ; \ \bar{x} + z_{\alpha/2} \sqrt{\frac{s^2}{n}}\bigg]$, where $\bar{x}$ is the sample average, $s^2$ is the sample variance, $z_{{\alpha}/{2}}$ is the ${\alpha}/{2}$ quantile of a standard normal distribution and $n$ is the sample size.

2. Find the value $\alpha^*$ for which the amplitude of $I(\mu, \alpha^*)$ is minimal and has at least one element in common with $\{-c, c\}$ (i.e., the border of $[-c,c]$). This $\alpha^*$ is the $s$-value.

3. On the one hand, if $\bar{x} \in [-c,c]$, then the observed sample is corroborating with the null Hypothesis $H_0: |\mu|\leq c$; if the $s$-value is small enough then you can accept the null. On the other hand, if $\bar{x} \not \in [-c,c]$, then the observed sample is providing information against the null Hypothesis $H_0$; if the $s$-value is small enough then you can reject the null. Otherwise, you should not reject or accept the null.

Notice that, if $\bar{x}\in [-c,c]$ and the respective $s$-value is extremely small, this means that the alternative hypothesis is extremely far away from the maximum plausible value, $\bar{x}$. If $\bar{x}\not \in [-c,c]$ and the respective $s$-value is extremely small, this means that the null hypothesis is extremely far away from the maximum plausible value, $\bar{x}$. Try to draw a picture representing the confidence interval and the null hypothesis of interest to better understand the conclusions. For more information please read the original paper Patriota (2013).

How to find objective thresholds for accepting or rejecting the null by using this $s$-value is still an open problem. This approach is nice because we can now accept a null hypothesis. This makes sense whenever the observed sample corroborates with the null and it is far away from the alternative. In your example it can be seen for $c = 1000$, $\bar{x} = 1$, $s^2 = 1$ and $n=10000$. It is quite simple to see that the data density is extremely concentrated on $[0.9, \ 1.1]$ (ten times the standard error). In order to have a non-empty intersection with $[-1000, \ 1000]$ it is required 99900 standard errors. Therefore, it would be fair enough to accept $H_0: |\mu| \leq c$ in this case.

References:

Patriota, A.G. (2013). A classical measure of evidence for general null hypotheses, Fuzzy Sets and Systems, 233, 74–88

Schervish, M.J. (1996). P Values: What they are and what they are not, The American Statistician, 50, 203–206.