# ANOVA for equivalence testing

How to perform a ANOVA for statistical equivalence?

I red about the two one-sided test (TOST) for equivalence, but (I think) for this study design it is not possible to perform a classical t-test, so a repeated measures ANOVA is needed.

The study design is a common pre-post treatment-control design: There are two different groups (control / treatment) and dependent two measure points at the baseline and a followup (MP1 / MP2). The research question is, if the treatments are equal.

My thought is, that the ANOVA analyses the differences between the groups. Is there a post-hoc for equivalence, special in R?

EDIT

Thanks to D_Williams, I red the using-lsmeans vignette and the lsmeans reference manual. There is a function called test. With this function you can do equivalence / noninferiority or nonsuperiority tests. Now I got stuck with a new problem. Because of the study design I need a linear regression with repeated measures. So here is a example dataset:

 dataset <- data.frame (ID    = rep(1:16),
GROUP = factor(rep(c("A","B"),8)),
MP1   = c(15,12,20,17,28,24,17,10,14,10,25,23,9,18,19,20),
MP2   = c(12,9,19,10,20,15,12,5,12,10,22,15,8,17,10,19),
)


The linear regression should be:

data.lm <- lm (MP2 - MP1 ~ GROUP, data=dataset)


As far as I understood this answer right, Case 2b is the correct one for me. Because there are two quantitative variables and a one dichotomous variable.

Afterwards I run these commands:

library("lme4","lsmeans","estimablity")
data.lsm <- lsmeans(data.lm,"GROUP")
test(data.lsm, null = log(100), delta = 0.20, side="equivalence")


Delta is the equivalence margin or the range of similarity. But I am not sure what the log(100) is.

My questions are:

1. Is this approach right?

and

2. To be honest: I don't really understand the last steps. Does anybody got some code example for studying them?

• What are MP1 & MP2 ? Are the control and treatment group made by the same units? – utobi Nov 12 '16 at 11:45
• MP1 and MP2 are the two dependent measure points at the beginning and at the end. The groups are not in the same unit. – M.Unterreiner Nov 14 '16 at 8:33
• Ok, but what do you mean by "equivalence" test? Are you interested in the mean/median values or on the entire distribution of MP1&MP2. Just to be sure I understood correctly, before answering to your question. – utobi Nov 15 '16 at 12:52
• Usually you test for superiority with the H1 hypothesis testing. Here, the hypothesis H1 is, that the control treatment and the "new" treatment is equal. In this case I am interested in the mean / medians values. – M.Unterreiner Nov 15 '16 at 13:43
• @M.Unterreinter I think you are making confusion about H0 and H1 hypothesis. Typically, H0 is the equality hypothesis and H1 is the alternative, which can be uni or two-directional. In your case, I suspect, you have H0: old treatment = new treatment, H1: old treatment different from new treatment (or old treatment worse than new treatment ). Is that so? – utobi Nov 15 '16 at 13:50

Since you have only two groups, there is no need to perform an ANOVA. You can perform a TOST procedure by simply building a confidence interval (CI) for a two sample problem, say x and y, where x = MP2-MP1 in the control group and y = MP2-MP1 in the treated group. You need to pay attention to the usual assumptions for the statistical tests: normality, heteroskedasticity, etc. But, once you have your CI, you can see if it is contained within the +-delta region or not. In R you can do something like this:

set.seed(12)
x = rnorm(100)
y = rnorm(70)

# for two sample t-test with equal variances
tt <- t.test(x, y, var.equal = TRUE, conf.level = 0.90)

# for Welch two sample t-test
tt.uneq <- t.test(x, y, var.equal = FALSE, conf.level = 0.90)

# for two sample Wilcoxon test
wcox <- wilcox.test(x,y, conf.int = TRUE, conf.level = 0.90)

delta <- 0.32

library(plotrix)
plotCI(x = 1, y=diff(rev(tt$estimate)), ui = tt$conf.int[2], li = tt$conf.int[1], xlim=c(0,4), ylab = "Confidence intervals", ylim = c(-0.6, 0.6), xlab ="Methods") plotCI(x = 2, y=diff(rev(tt.uneq$estimate)), ui = tt.uneq$conf.int[2], li = tt.uneq$conf.int[1], add=TRUE, col=2)
plotCI(x = 3, y=wcox$estimate, ui = wcox$conf.int[2],
li = wcox\$conf.int[1], add=TRUE, col=3)
abline(h = c(-delta, delta), lwd = 2, lty = 2)


Here, to illustrate the idea I'm considering three type of tests but there are many others in the literature. Hope this helps.

• Thank you really much utobi. I thought that you can't use a t-test with repeated measurements and two groups because of the Type I error. Is it right, that with a CI there won't be this kind of error? – M.Unterreiner Nov 16 '16 at 20:10
• M.Unterreiner, the type I error rate is ubiquitous in statistical testing. Having said that, nothing special with repeated measures. In such a case, you have balanced samples so you can use either t.test or wilcox.test but you have to specify the option "paired=TRUE". – utobi Nov 17 '16 at 8:29
• What if you also wanted to test that the increases (MP2-MP1) for each condition were significant? ie. MP2 was significantly different from MP1 in each condition. How would you test for all of these without inflating your Type-1 error rate? – strava Jul 26 at 19:21

Generally equivalence testing is very limited to simple models. I have, however, recently learned that the lsmeans package in r does equivalence testing for models more complex than a t-test. I have not used it, but I have used lsmeans and it is a very flexible tool. This might work, but you will have to check out the package vignette for the details to be sure see here