# ANOVA for equivalence testing

How to perform a ANOVA for statistical equivalence?

I read about the two one-sided test (TOST) for equivalence, but (I think) for this study design it is not possible to perform a classical t-test, so a repeated measures ANOVA is needed.

The study design is a common pre-post treatment-control design: there are two different groups (control / treatment) and dependent two measure points at the baseline and a followup (MP1 / MP2). The research question is, if the treatments are equal.

My thought is, that the ANOVA analyses the differences between the groups. Is there a post-hoc for equivalence, special in R?

EDIT

Thanks to D_Williams, I read the using-lsmeans vignette and the lsmeans reference manual. There is a function called test. With this function you can do equivalence / noninferiority or nonsuperiority tests. Now I got stuck with a new problem. Because of the study design I need a linear regression with repeated measures. So here is a example dataset:

 dataset <- data.frame (ID    = rep(1:16),
GROUP = factor(rep(c("A","B"),8)),
MP1   = c(15,12,20,17,28,24,17,10,14,10,25,23,9,18,19,20),
MP2   = c(12,9,19,10,20,15,12,5,12,10,22,15,8,17,10,19),
)


The linear regression should be:

data.lm <- lm (MP2 - MP1 ~ GROUP, data=dataset)


As far as I understood this answer right, Case 2b is the correct one for me. Because there are two quantitative variables and a one dichotomous variable.

Afterwards I run these commands:

library("lme4", "lsmeans", "estimablity")
data.lsm <- lsmeans(data.lm, "GROUP")
test(data.lsm, null = log(100), delta = 0.20, side="equivalence")


Delta is the equivalence margin or the range of similarity. But I am not sure what the log(100) is.

My questions are:

1. Is this approach right?

and

2. To be honest: I don't really understand the last steps. Does anybody have some code example for studying them?

• What are MP1 & MP2 ? Are the control and treatment group made by the same units? Nov 12, 2016 at 11:45
• MP1 and MP2 are the two dependent measure points at the beginning and at the end. The groups are not in the same unit. Nov 14, 2016 at 8:33
• Ok, but what do you mean by "equivalence" test? Are you interested in the mean/median values or on the entire distribution of MP1&MP2. Just to be sure I understood correctly, before answering to your question. Nov 15, 2016 at 12:52
• Usually you test for superiority with the H1 hypothesis testing. Here, the hypothesis H1 is, that the control treatment and the "new" treatment is equal. In this case I am interested in the mean / medians values. Nov 15, 2016 at 13:43
• @M.Unterreinter I think you are making confusion about H0 and H1 hypothesis. Typically, H0 is the equality hypothesis and H1 is the alternative, which can be uni or two-directional. In your case, I suspect, you have H0: old treatment = new treatment, H1: old treatment different from new treatment (or old treatment worse than new treatment ). Is that so? Nov 15, 2016 at 13:50

Since you have only two groups, there is no need to perform an ANOVA. You can perform a TOST procedure by simply building a confidence interval (CI) for a two sample problem, say x and y, where x = MP2-MP1 in the control group and y = MP2-MP1 in the treated group. You need to pay attention to the usual assumptions for the statistical tests: normality, heteroskedasticity, etc. But, once you have your CI, you can see if it is contained within the +-delta region or not. In R you can do something like this:

set.seed(12)
x = rnorm(100)
y = rnorm(70)

# for two sample t-test with equal variances
tt <- t.test(x, y, var.equal = TRUE, conf.level = 0.90)

# for Welch two sample t-test
tt.uneq <- t.test(x, y, var.equal = FALSE, conf.level = 0.90)

# for two sample Wilcoxon test
wcox <- wilcox.test(x,y, conf.int = TRUE, conf.level = 0.90)

delta <- 0.32

library(plotrix)
plotCI(x = 1, y=diff(rev(tt$estimate)), ui = tt$conf.int[2], li = tt$conf.int[1], xlim=c(0,4), ylab = "Confidence intervals", ylim = c(-0.6, 0.6), xlab ="Methods") plotCI(x = 2, y=diff(rev(tt.uneq$estimate)), ui = tt.uneq$conf.int[2], li = tt.uneq$conf.int[1], add=TRUE, col=2)
plotCI(x = 3, y=wcox$estimate, ui = wcox$conf.int[2],

As of 2022, the emmeans package (successor to lmmeans can be used for equivalence tests of linear models; vignette info is here.