Contradictory test results My question is relatively simple, and has to do with reporting seemingly contradictory results from statistical tests. This is the problem.
There are two independent groups, and two research questions are important:

*

*Are the two (population) group means equal

*Do both (population) means differ significantly from zero?

An independent groups t-test leads to the conclusion (A) that both group means do NOT differ significantly from one another. Further, two “one sample t-tests”, lead to the conclusion (B) that group-mean 1 differs significantly from zero, whereas group-mean 2 does NOT.
These two test conclusions seem contradictory, because for the true means in the populations conclusion (A) and (B) could not both be valid. Sample tests, of course, can lead to such contradictory conclusions. Hence my question is: how to deal with such results, or how to report them? Simply reporting both conclusions (A) and (B) is not "wrong" but it feels inconsistent. Any thoughts about this?
 A: This is indeed a problem (one of many) with hypothesis testing. This particular scenario is discussed in detail in the paper by Andrew Gelman and Hal Stern titled The Difference Between "Significant" and "Not Significant" is not Itself Statistically Significant
From the abstract:

The ubiquity of this statistical error leads us to suggest that students and practitioners be made more aware that the difference between “significant” and “not significant” is not itself statistically significant.

From the discussion:

Statistical significance, in some form, is a way to assess the reliability of statistical findings. However, as we have seen, comparisons of the sort, “X is statistically significant but Y is not,” can be misleading.

A: Consider the simplified case of intersecting intervals (see below). It can be the case to have three intervals such that A and C both overlap with B, but A and C don't overlap with each other.

To translate this to hypothesis testing, think of the test as of looking if confidence intervals overlap.
A: It is a completely acceptable situation that can happen easily. Informally, (and ignoring volatility differences), it just means that both group sample means are not that far apart, but the group 2 sample mean is a bit closer to 0 than is the group 1 sample mean so they fall on different sides of their respective p-value thresholds. Say, the group 2 mean is 1.8 and the group 1 mean is 2.2. The sample difference is just 0.4, much closer to 0 than either of its component means. That's just the nature of testing and sample variability.
There is an important fact about the volatility of differences that impacts on this. The variance of the difference of two independent sample means is the sum of the individual variances. So a difference is much more volatile than its components, requiring bigger differences to achieve low p-values!
