TOST, an acronym for *two one-sided tests* is a straightforward way of constructing a test of the "negativist" null hypothesis that two population statistics differ by no more than a small researcher-selected equivalence threshold.

Hypothesis tests are most commonly framed in terms of null hypotheses of no difference (e.g. two parameters are equal, and their difference is zero, or their ratio is one). In such a case an extreme enough test statistic (i.e. one which is improbable if the the null hypothesis is true) provides evidence that the null hypothesis is false, and one concludes the parameters are different.

By contrast, one may wish to frame a null hypothesis of difference at least as large as a given level, and here an extreme enough test statistic provides evidence to reject this null hypothesis, and conclude that the parameters are equivalent within the given tolerance. With respect to t and z type tests, the general form of the negativist null hypothesis is $\text{H}^{-}_{0}\text{: }|\theta| \ge \Delta$, which takes the specific form to two one-sided null hypotheses: $\text{H}^{-}_{01}\text{: }\theta \ge \Delta$ or $\text{H}^{-}_{02}\text{: }\theta \le -\Delta$. If one rejects $\text{H}^{-}_{01}$, then $\theta$ must be less than $\Delta$, and if one rejects $\text{H}^{-}_{02}$ then $\theta$ must be greater than $-\Delta$. If one rejects both these one-sided null hypotheses, then $-\Delta < \theta < \Delta$.

The two t test statistics corresponding to these specific null hypotheses are (the corresponding z test statistics would naturally use $\sigma_{\theta}$):

  1. $t_{1} = \frac{\Delta - \theta}{s_{\theta}}$

  2. $t_{2} = \frac{\theta + \Delta}{s_{\theta}}$

The rejection regions for both these statistics are in the right tail, and both tests must be rejected in order to conclude equivalence. The probability of a Type I error is made by conducting both tests at the $\alpha$ level, rather than the $\alpha/2$ level, because the rejection regions of the null hypotheses are non-overlapping.

The equivalence threshold $\Delta$ is expressed in the same units as the measures being tested. However, it may be desirable to express equivalence in terms of the test statistic itself, and this can be done by using $\varepsilon$, where $\varepsilon=\Delta/s_{\theta}$. The meaning of the equivalence threshold when using $\varepsilon$ is then "how far past the rejection boundary for the test for difference $\text{H}^{+}_{0}$ a test statistic needs to be to be considered relevant." In this case, $\text{H}^{-}_{0}\text{: }|t| \ge \varepsilon$, so that $\text{H}^{-}_{01}\text{: }t \ge \varepsilon$, $\text{H}^{-}_{02}\text{: }t \le -\varepsilon$, and $t_{1} = \varepsilon - t$ and $t_{2} = t + \varepsilon$, where $t=\theta/s_{\theta}$. Note that if $\varepsilon \le t_{1-\alpha}$, then it is not possible to reject any $\text{H}^{-}_{0}$, because either $t_{1}$ or $t_{2}$ will be less than or equal to zero.

history | excerpt history