# Is significance more important than absolute value?

I have three groups and I'm comparing two graphs in each group using the t-test: the absolute difference between two graphs in the first group is 34 with a p=0.024. (*) The absolute difference in the second group is 28 with a p=0.04 (*) and in the third group the absolute difference is 4 but the p value is=0.010 (**)

How do I explain this data? The absolute value in the first and second group is higher; however, the significance is less than the comparison in group three where the absolute value difference between two groups is only 4. So, should I say there was more change in group three (**) than in group 1 and 2 although the absolute difference (4) is less in this group?

Thank you.

• I'm not quite sure what you are comparing - what are the "graphs" and how are you comparing them? A t-test compares ACROSS two groups: E.g if the groups are "men" and "women" and you are comparing their heights; it looks at differences between the groups. You have 3 groups; are you comparing across the three groups? Then why not use ANOVA rather than t-tests? – Peter Flom Sep 13 '11 at 9:48

The p-value is a nonlinear tranformation of the difference and variation. This makes it very hard to compare p-values to other p-values meaningfully (other than to say which are significant and which are not significant).

Much better is to compute confidence intervals which show both the absolute difference and the variation around the estimated difference. While the difference of 34 is the largest, the end of the interval may be closer to 0 than that for the difference of 4 showing that while your estimate is bigger, the variation means that it could in fact be smaller.

• Just a little question: Do I have to calculate 95%CI for Means or for the difference between the Means? How to calculate the difference? Thank you – starkid Sep 12 '11 at 23:09
• If you are comparing groups then you should compute the confidence intervals of the differences. The formulas are in most intro stats text books and are implemented in any decent statistics package (t.test function in R is one example). – Greg Snow Sep 13 '11 at 16:15

The P value from the t test depends upon:

• The absolute value of the difference between the means
• Variability, as assessed by the SD of the two groups
• Sample size

So the apparent discrepancy in your results must be the result of very different SDs among the groups, or different sample sizes.

Note that the t test assumes that both sets of data are sampled from Gaussian populations with equal standard deviations.

• is right about the assumptions, but there are variations of the t-test that relax the assumption about equal variances (or standard deviations). SAS gives the Satterthwaite approximations for such cases. – Peter Flom Sep 13 '11 at 9:49