10
$\begingroup$

In t-tests (paired or independent) we test if the difference in means if 0 (or another value). I'm wondering if it's possible to instead test if the difference is less than x?

$\endgroup$

1 Answer 1

13
$\begingroup$

Sure, you can do that. You don't have to test against a null hypothesis of $0$ (sometimes called a "nil null"); you can test against any value. You also don't have to do a two-tailed test; you can perform a one-tailed test (when specified a-priori). The paired $t$-test is:
$$ t = \frac{\bar x_D - \mu_{\rm null}}{\frac{s_D}{\sqrt N}} $$ Thus, to combine the two less typical possibilities noted above, you substitute your specific value for $\mu_{\rm null}$, and run a one-tailed test.

Here is a simple example (coded in R):

set.seed(2786)                     # this makes the example exactly reproducible
x1 = rnorm(20, mean=3, sd=5)       # I'm generating data from a normal distribution
x2 = x1 - rnorm(20, mean=0, sd=1)  # the true difference is 0
## this is a paired t-test of whether the difference is <1:
t.test(x1, x2, mu=1, alternative="less", paired=TRUE)
# 
#  Paired t-test
# 
# data:  x1 and x2
# t = -7.5783, df = 19, p-value = 1.855e-07
# alternative hypothesis: true difference in means is less than 1
# 95 percent confidence interval:
#         -Inf -0.02484498
# sample estimates:
# mean of the differences 
#              -0.3278085 
$\endgroup$
9
  • $\begingroup$ Thank you! I'm really wishing I took stats more seriously in university. I'm finding it so useful and interesting now that I'm actually applying it. If I want to show that $x$ and $y$ are very similar within 1 units of measurement, would I use the same values you just did? (alternative is that the true mean is less than 1 instead of the alternative being the true mean is greater than 1) $\endgroup$
    – DeanAttali
    Apr 8, 2016 at 23:15
  • 3
    $\begingroup$ Yes, that's called two one-sided tests. Questions on CV on this topic can be found by searching on tost. You may also be interested in reading my answer here: Why do statisticians say a non-significant result means “you can't reject the null” as opposed to accepting the null hypothesis? $\endgroup$ Apr 8, 2016 at 23:19
  • $\begingroup$ Thank you for that. That's a great resource. My next question was going to be if that tost can be paired, and after finding an R package for it, it seems like it does have that parameter! $\endgroup$
    – DeanAttali
    Apr 8, 2016 at 23:52
  • 1
    $\begingroup$ @daattali please don't ask question after question in comments when you're not clarifying something relating to the meaning of the answer to the actual question you asked about, but asking a new question that arises because of understanding you gain from the answer (the first is fine, the second is not). Equivalence tests (which are generally done via TOST) are a whole subject, and many posts on site relate to them. Read about them and then if you have a question not already answered on site, post a new question. $\endgroup$
    – Glen_b
    Apr 9, 2016 at 1:36
  • 2
    $\begingroup$ @Glen_b I thought my question is directly related to the original question, because I wanted to know how TOST (which was suggested in the comments) related to the original question+answer. I now understand that a TOST is really just performing the t-test in the answer twice, and if both null hypotheses are rejected, then the null hypothesis of the TOST is rejected --> conclude equivalence. That's the missing link that I was trying to understand. $\endgroup$
    – DeanAttali
    Apr 9, 2016 at 5:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.