Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

As title, I have two set of data that are identical and performed paired t-test using R, the p-value for the difference estimated (which is essentially 0) is NaN. I wonder if there can be a theoretical p-value for this case.

share|improve this question

2 Answers 2

up vote 9 down vote accepted
  1. If the two sets of data are identical because the variables are defined on a discrete set of values, then the assumptions of the t-test are false, since the random variables aren't even continuous.

    As such, any normal-theory calculation would not yield the correct p-values.

    I think the "correct" t-test p-value would be NaN.

  2. Alternatively, consider the degenerate case where continuous variables have finite variance but the differences have variance 0. Then the t-statistic would be 0/0; again I'd say that's "correctly" NaN. (edit: more details of that argument are in a comment)

  3. However, if you (for example) assumed some discrete distribution for the differences and derived say a likelihood ratio test, or did a permutation test, you'd get a (legitimate) p-value of 1, since anything but zero-differences would be "more extreme" than all-0-differences.

So I think justifiable p-values for a t-test will actually be what R gave you, while justifiable p-values if you have discrete variables and choose a more appropriate test would generally be 1.

share|improve this answer

The null hypothesis for a paired t-test is that the mean difference of your two paired samples is equal to zero.

Considering you have identical values for each pair, the difference should always be equal to zero, thus failing to reject the null hypothesis.

This would mean p-value = 1 or NaN (0/0), but this doesn't really imply you should conclude anything.

A p-value of less that 0.05 will suggest the existence of a significantly different mean for each sample.

share|improve this answer
2  
Maybe it is better to say a small p-value suggests since 0.05 is entirely arbitrary (even though it is the de facto standard cutoff). –  Marc Claesen Aug 11 at 7:59
5  
I don't think we can really say the p-value for the t-test must be 1, since the t-statistic itself ($\frac{d}{s_d/\sqrt{n}}$) is actually 0/0. You can construct sequences of Gaussian (vector-) random variables representing the differences where the t-statistic is constant but the numerator and denominator converge to 0, which means that either the statistic is undefined in the limit (since it's consistent with a sequence converging to literally any finite value) or you're in a situation where the t-test assumptions don't hold. –  Glen_b Aug 11 at 8:07
1  
Arun, I'm saying that I think the interpretation is incorrect (especially with the word "must" in there). Is there a flaw in the explanation that $t=0/0$ is consistent with sequences of variables that would yield any particular t-value? You really can't treat 0/0 as zero merely by ignoring the denominator. It would require a stronger argument than that, I think. What is the population situation you're positing which yields a mean pair-difference of 0? –  Glen_b Aug 11 at 8:15
1  
Your argument appears to be "if the differences hadn't all been zero, yet the mean difference was still zero, then the p-value would be 1". This justifies a p-value of 1 for a different observed set of differences than we have. Surely the question seeks a justification of a p-value for the situation at hand, rather than one for a different situation. –  Glen_b Aug 11 at 8:34
2  
Ain't no "confirming the null hypothesis", just failing to reject it. For example, a small sample size with discrete (or low-precision) data could lead to the OP's condition, rather than actual population equality between groups. Best not to conflate failing to reject the null with evidence for equality or evidence for equivalence. –  Alexis Aug 11 at 13:35

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.