I am using scipy in Python and the following return a nan value for whatever reason:

>>>stats.ttest_ind([1, 1], [1, 1])
Ttest_indResult(statistic=nan, pvalue=nan)

>>>stats.ttest_ind([1, 1], [1, 1, 1])
Ttest_indResult(statistic=nan, pvalue=nan).

But whenever I use samples that have different summary statistics, I actually get a reasonable value:

stats.ttest_ind([1, 1], [1, 1, 1, 2])
Ttest_indResult(statistic=-0.66666666666666663, pvalue=0.54146973927558495).

Is it reasonable to interpret a p-value of nan as 0 instead? Is there any reason from statistics that it doesn't make sense to run a 2-sample t-test on samples with the same summary statistics?

  • 3
    $\begingroup$ Assuming that the arguments of ttest_ind are lists of raw data, rather than summary statistics, then I'd assume the problem is due to the first two examples having no variance in either variable. This may potentially be a question best asked on Stack Overflow though $\endgroup$ – Ian_Fin Jul 13 '16 at 15:37
  • $\begingroup$ Yeah, I get a nan whenever I compare samples that have no variance. I was just wondering if it somehow makes no sense to do so from the point of view of statistics. I'll ask this question on StackOverflow, too. $\endgroup$ – bourbaki4481472 Jul 13 '16 at 15:42
  • 2
    $\begingroup$ No, it makes no sense. Statistical tests often effectively boil down to accounting for variance. You have no variance, so nothing to account for. $\endgroup$ – Ian_Fin Jul 13 '16 at 15:45
  • $\begingroup$ Okay, so I'll have to handle this special case in my larger program. $\endgroup$ – bourbaki4481472 Jul 13 '16 at 15:56

The problem with trying to compare two constant samples with a t-test is that the calculation of t involves an estimate of within-group SD in its denominator. From Wikipedia:

$$t = \frac{\bar {X}_1 - \bar{X}_2}{s_{X_1X_2} \cdot \sqrt{\frac{1}{n_1}+\frac{1}{n_2}}}$$

When both samples are constant, $s_{X_1X_2} = 0$, leading to a division by 0.

  • $\begingroup$ Is it correct to just say that the p-value in this case is 0? $\endgroup$ – bourbaki4481472 Jul 13 '16 at 15:56
  • $\begingroup$ No. $t$ is undefined, so you don't have a $p$-value for this test. $\endgroup$ – Kodiologist Jul 13 '16 at 15:57
  • $\begingroup$ What about interpreting the p-value as the probability of rejecting the null hypothesis? Surely, I can say that two populations are the same if they have the same variance AND the same mean? $\endgroup$ – bourbaki4481472 Jul 13 '16 at 16:00
  • $\begingroup$ That's not what a $p$-value is; a $p$-value is the probability under the null hypothesis of getting a test statistic at least as extreme as the one you obtained. $\endgroup$ – Kodiologist Jul 13 '16 at 16:06
  • $\begingroup$ Got it. I think I will leave the nan in my program then. $\endgroup$ – bourbaki4481472 Jul 13 '16 at 16:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.