I'm looking at Lakens' (2017) primer and he tests the hypothesis that there is a difference between two groups $X_1$and $X_2$of magnitude $\Delta=E[X_1]-E[X_2] \neq0$ by subtracting $\Delta$ from the difference between the sample means $d=M_1-M_2$ and then he computes the $t$ value and $p$ value based on this difference score with a procedure analogous to Welch t-test (Eqs 3 and 4 on p. 357).

This looks wrong to me. IMO to determine the p value, one should be using non-central $t$ distribution with non-centrality parameter based on $\Delta$ and t value based on $d$. This is because under the hypothesis, $d$ and hence the test statistic $t=d/h$ (where we would substitute $h=\sqrt{s_1^2/n_1+s_2^2/n_2}$ following Welch) does not have a symmetric distribution because $E[X_1]-E[X_2]=\Delta\neq 0$. Subtracting $\Delta$ from $d$ does merely shift the distribution, but it can't make it symmetric, since $\Delta$ is not a random variable.

Another CrossValidated answer claims that

Using the zero centered T distribution here would test the hypothesis assuming that X1 - X2 -3 is symmetric about zero.`

(X1-X2-3 corresponds to $X_1-X_2-\Delta$). I don't see how it is possible to obtain such symmetric test statistic distribution unless $E[X_1]=E[X_2]$ which under the hypothesis is not the case ($E[X_1]-E[X_2]=\Delta\neq 0$).

So, which is the correct procedure to conduct equivalence test?


Lakens, D. (2017). Equivalence tests: a practical primer for t tests, correlations, and meta-analyses. Social Psychological and Personality Science, 8(4), 355-362.


What you are proposing does not appear to be any different from what actually occurs in this test. Remember that in a classical hypothesis test, the p-value is calculated using the null distribution of the test statistic. This calculation assumes that the null hypothesis is true. You suggest that the calculation of the p-value should use the non-central T-distribution with non-centrality parameter $\Delta$. However, the null hypothesis for this test is that $\Delta = 0$, so under that condition the non-central T-distribution simplifies down to the standard (centralised) T-distribution.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.