0
$\begingroup$

I have a repeated measures experiment design with two independent variables ('condition' with 2 levels and 'repeat' with 10 levels) and one dependent variable ('performance'). I'd like to know whether condition A and condition B result in equivalent performance.

My question is similar to this one: ANOVA for equivalence testing, except that I have a paired design (i.e. 'condition' is also a repeated measure).

I think some sort of TOST procedure would be appropriate, but I am not sure what test I should use within the procedure. Do I need to use a two-way repeated measures ANOVA? If so, is it possible to construct a confidence interval using such an ANOVA test?

| cite | improve this question | |
$\endgroup$
  • $\begingroup$ What does "repeat" refer to? Does this mean that you have 20 measurements of "performance" per person: 10 repeats in Condition A and 10 repeats in Condition B? And your research question is: "Do people perform differently in Condition A and Condition B?" $\endgroup$ – Mark White Apr 19 '18 at 17:04
  • $\begingroup$ Yes. To give some context: in my experiments the unit of analysis is a machine learning task (dataset) and the performance refers to the performance of an algorithm in either condition A or condition B. Because there is some randomness involved in the algorithm, I repeat the experiment 10 times (so 10 times A and 10 times B) with a different random seed. So for each task ('person') I have measurements for combinations A-1, A-2, ..., A-10, B-1, B-2, ..., B-10. $\endgroup$ – Hilde Apr 19 '18 at 18:46
  • $\begingroup$ The research question is: "Is the performance of tasks in condition A equivalent to performance of tasks in condition B?". So I don't want to know whether the performance is different; I want to know whether the performance is equivalent given some equivalence margin. $\endgroup$ – Hilde Apr 19 '18 at 18:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.