In t-tests (paired or independent) we test if the difference in means if 0 (or another value). I'm wondering if it's possible to instead test if the difference is less than x?
1 Answer
Sure, you can do that. You don't have to test against a null hypothesis of $0$ (sometimes called a "nil null"); you can test against any value. You also don't have to do a two-tailed test; you can perform a one-tailed test (when specified a-priori). The paired $t$-test is:
$$
t = \frac{\bar x_D - \mu_{\rm null}}{\frac{s_D}{\sqrt N}}
$$
Thus, to combine the two less typical possibilities noted above, you substitute your specific value for $\mu_{\rm null}$, and run a one-tailed test.
Here is a simple example (coded in R
):
set.seed(2786) # this makes the example exactly reproducible
x1 = rnorm(20, mean=3, sd=5) # I'm generating data from a normal distribution
x2 = x1 - rnorm(20, mean=0, sd=1) # the true difference is 0
## this is a paired t-test of whether the difference is <1:
t.test(x1, x2, mu=1, alternative="less", paired=TRUE)
#
# Paired t-test
#
# data: x1 and x2
# t = -7.5783, df = 19, p-value = 1.855e-07
# alternative hypothesis: true difference in means is less than 1
# 95 percent confidence interval:
# -Inf -0.02484498
# sample estimates:
# mean of the differences
# -0.3278085
-
$\begingroup$ Thank you! I'm really wishing I took stats more seriously in university. I'm finding it so useful and interesting now that I'm actually applying it. If I want to show that $x$ and $y$ are very similar within 1 units of measurement, would I use the same values you just did? (alternative is that the true mean is less than 1 instead of the alternative being the true mean is greater than 1) $\endgroup$ Apr 8, 2016 at 23:15
-
3$\begingroup$ Yes, that's called two one-sided tests. Questions on CV on this topic can be found by searching on tost. You may also be interested in reading my answer here: Why do statisticians say a non-significant result means “you can't reject the null” as opposed to accepting the null hypothesis? $\endgroup$ Apr 8, 2016 at 23:19
-
$\begingroup$ Thank you for that. That's a great resource. My next question was going to be if that tost can be paired, and after finding an R package for it, it seems like it does have that parameter! $\endgroup$ Apr 8, 2016 at 23:52
-
1$\begingroup$ @daattali please don't ask question after question in comments when you're not clarifying something relating to the meaning of the answer to the actual question you asked about, but asking a new question that arises because of understanding you gain from the answer (the first is fine, the second is not). Equivalence tests (which are generally done via TOST) are a whole subject, and many posts on site relate to them. Read about them and then if you have a question not already answered on site, post a new question. $\endgroup$– Glen_bApr 9, 2016 at 1:36
-
2$\begingroup$ @Glen_b I thought my question is directly related to the original question, because I wanted to know how TOST (which was suggested in the comments) related to the original question+answer. I now understand that a TOST is really just performing the t-test in the answer twice, and if both null hypotheses are rejected, then the null hypothesis of the TOST is rejected --> conclude equivalence. That's the missing link that I was trying to understand. $\endgroup$ Apr 9, 2016 at 5:04