0
$\begingroup$

I am using scipy in Python and the following return a nan value for whatever reason: 

>>>stats.ttest_ind([1, 1], [1, 1])
Ttest_indResult(statistic=nan, pvalue=nan)

>>>stats.ttest_ind([1, 1], [1, 1, 1])
Ttest_indResult(statistic=nan, pvalue=nan).

But whenever I use samples that have different summary statistics, I actually get a reasonable value: 

stats.ttest_ind([1, 1], [1, 1, 1, 2])
Ttest_indResult(statistic=-0.66666666666666663, pvalue=0.54146973927558495).

Is it reasonable to interpret a p-value of nan as 0 instead? Is there any reason from statistics that it doesn't make sense to run a 2-sample t-test on samples with the same summary statistics?

Improve this question
$\endgroup$
4
  • 3
    $\begingroup$ Assuming that the arguments of ttest_ind are lists of raw data, rather than summary statistics, then I'd assume the problem is due to the first two examples having no variance in either variable. This may potentially be a question best asked on Stack Overflow though $\endgroup$
    – Ian_Fin
    Commented Jul 13, 2016 at 15:37
  • $\begingroup$ Yeah, I get a nan whenever I compare samples that have no variance. I was just wondering if it somehow makes no sense to do so from the point of view of statistics. I'll ask this question on StackOverflow, too. $\endgroup$
    – under_the_sea_salad
    Commented Jul 13, 2016 at 15:42
  • 2
    $\begingroup$ No, it makes no sense. Statistical tests often effectively boil down to accounting for variance. You have no variance, so nothing to account for. $\endgroup$
    – Ian_Fin
    Commented Jul 13, 2016 at 15:45
  • $\begingroup$ Okay, so I'll have to handle this special case in my larger program. $\endgroup$
    – under_the_sea_salad
    Commented Jul 13, 2016 at 15:56

1 Answer 1

Reset to default
4
$\begingroup$

The problem with trying to compare two constant samples with a t-test is that the calculation of t involves an estimate of within-group SD in its denominator. From Wikipedia:

$$t = \frac{\bar {X}_1 - \bar{X}_2}{s_{X_1X_2} \cdot \sqrt{\frac{1}{n_1}+\frac{1}{n_2}}}$$

When both samples are constant, $s_{X_1X_2} = 0$, leading to a division by 0.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Is it correct to just say that the p-value in this case is 0? $\endgroup$
    – under_the_sea_salad
    Commented Jul 13, 2016 at 15:56
  • $\begingroup$ No. $t$ is undefined, so you don't have a $p$-value for this test. $\endgroup$
    – Kodiologist
    Commented Jul 13, 2016 at 15:57
  • $\begingroup$ What about interpreting the p-value as the probability of rejecting the null hypothesis? Surely, I can say that two populations are the same if they have the same variance AND the same mean? $\endgroup$
    – under_the_sea_salad
    Commented Jul 13, 2016 at 16:00
  • $\begingroup$ That's not what a $p$-value is; a $p$-value is the probability under the null hypothesis of getting a test statistic at least as extreme as the one you obtained. $\endgroup$
    – Kodiologist
    Commented Jul 13, 2016 at 16:06
  • $\begingroup$ Got it. I think I will leave the nan in my program then. $\endgroup$
    – under_the_sea_salad
    Commented Jul 13, 2016 at 16:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.