0
$\begingroup$

I need to state that there is no difference between the outcomes of the two tests (method 1 = fast and simple, method 2 = slow and hard to reproduce). I thought that a non-significant t-test would be enough, but I was wrong (can't affirm that a non-significant t-test means equivalence). So I've found the TOST (two one-sided t-tests) or equivalence test. Anyway, I'm struggling to get it correctly.

The TOST test examines the means, standard deviation and effect size (Cohen's d) to state that the values are equivalent. But, since I have almost the same results between method 1 and method 2, the Cohen's d will be obviously lower, so it does not allow me to compute the equivalence.

Am I doing something wrong?

mean_1=43.08649, mean_2=42.59865, sd_1=8.060118, sd_2=8.441185, n_1=37, n_2=37

Improve this question
$\endgroup$
6
  • 2
    $\begingroup$ What is the minimum difference between method 1 and method 2 which you find relevant? In other words: what is the boundary of "difference too small to care about" and "meaningful difference"? $\endgroup$
    – Alexis
    Dec 2, 2021 at 18:21
  • $\begingroup$ Is it something that I have to decide? I'm having troubles to understand this. Should I decide the interval I state that this assumption is true or not? Something like "Ok, for me a difference of + - 1 is not relevant"? $\endgroup$
    – Fede R
    Dec 2, 2021 at 19:31
  • $\begingroup$ Yes, exactly! The equivalence threshold is a researcher choice. See my answer to this question. $\endgroup$
    – Alexis
    Dec 2, 2021 at 19:42
  • $\begingroup$ Cool, thank you. Could you give me another help? Check me out. From the previous data, I'm placing my boundaries at +/- 0.5 so I get (with R): The equivalence test was significant, t(71.85) = -1.896, p = 0.031, given equivalence bounds of -4.126 and 4.126 (on a raw scale) and an alpha of 0.05. So, based on these results, I can state that my data are "statistically equivalent to 0" because the TOST p-value is < 0.05 (My alpha set).. right? $\endgroup$
    – Fede R
    Dec 2, 2021 at 20:22
  • $\begingroup$ The equivalence threshold is part of your findings. So "I found evidence the difference between method 1 & method two was equivalent within an equivalence range of $\pm0.5$ at the $\alpha=0.05$ level. $\endgroup$
    – Alexis
    Dec 2, 2021 at 23:04

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.