1
$\begingroup$

Introduction

I find the independent sample t-test to be one of the most misunderstood concepts in statistics. It is used to test for a difference between two population means by comparing the hypothesized difference between the two population means $(\mu_1-\mu_2)$ with the observed sample mean difference $(\overline{x}_{1}-\overline{x}_{2})$.

The test statistic is usually calculated as follows:

t = $\frac{(\overline{x}_{1}-\overline{x}_{2})-(\mu_1-\mu_2)}{SE}$

where:

$(\overline{x}_{1}-\overline{x}_{2})$ is the observed difference between sample means,

$(\mu_1-\mu_2)$ is the hypothesized difference between population means, and SE is the standard error, the calculation of which varies based on whether we assume equal variance or not.

The hypothesized difference in population means can be a value the researcher believes is true or even a standard value. A common standard value is 0, which is used when we want to hypothesize no difference in population means. In this special case, the test statistic formula simplifies to:

t = $\frac{(\overline{x}_{1}-\overline{x}_{2})}{SE}$

Many publications present this special case as the only way to calculate the test statistics, ignoring the fact that researchers can hypothesize the difference between population means to be a value different from 0.

problem

For Cohen's d calculation, I often see most publications using:

d = $\frac{(\overline{x}_{1}-\overline{x}_{2})}{sd}$

I am unsure whether I should use this formula when the hypothesized mean difference $(\mu_1-\mu_2)$ is a value different from zero, or if I should modify it to:

d = $\frac{(\overline{x}_{1}-\overline{x}_{2})-(\mu_1-\mu_2)}{sd}$

I feel with d = $\frac{(\overline{x}_{1}-\overline{x}_{2})}{sd}$ we might overlook the possibility that $(\mu_1-\mu_2)$ is not always zero.

$\endgroup$
1
  • $\begingroup$ Do you wish to interpret $d$ as the effect size, period, or as an effect size relative to the hypothesized effect size? Clearly these are equivalent ways of stating the same thing, so the key is to be clear about your meaning. But which expression you use is not statistically or mathematically relevant. $\endgroup$
    – whuber
    Commented Nov 22, 2023 at 15:49

1 Answer 1

1
$\begingroup$

You are correct that this could be a problem. The calculation for $d$ should be the way you modify it.

In defense of the publications that use the simpler form by setting $\mu_1 - \mu_2$ to 0, I find that this is the test that people want in the vast majority of cases. But it may be different in different fields.

$\endgroup$
2
  • 1
    $\begingroup$ Thank you. I believe there is a need for statisticians to always avoid such oversights. It is becoming a problem, especially when carried over by software developers. I have noticed several software programs making this mistake, one of which is the rstatix package. $\endgroup$ Commented Nov 22, 2023 at 13:27
  • $\begingroup$ I suggest you write to the maintainer of the package with your concern. $\endgroup$
    – Peter Flom
    Commented Nov 22, 2023 at 14:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.