5
$\begingroup$

I am finding that there is no significant difference between a number list and another list which has 100*same numbers. The P value is 0.4487. The means are obviously very different (100 times). Why is it so?

Following is code and data in R:

> t.test(vnum, 100*vnum)

        Welch Two Sample t-test

data:  vnum and 100 * vnum
t = -0.7637, df = 49.01, p-value = 0.4487
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -41.72902  18.74726
sample estimates:
 mean of x  mean of y 
 0.1160695 11.6069527 

vnum
 [1] -0.895220890  0.472985786 -1.153668533  0.299671810 -0.507962145  1.126608614 -1.964113989 -0.102483497  0.869147815
[10]  0.124949840  0.007075569  1.050399452  0.127889864  0.495323100 -1.442668978 -0.617975264  1.142867644 -1.423523025
[19]  0.426532832 -0.186958315  1.648847799 -0.460224243  1.654329219  0.740546166 -0.558588185  2.876850041  0.829542673
[28] -0.409403714  0.226379412  2.722903972  0.664555431  0.661642150  0.479198746 -0.188802783  0.667357712  0.642126933
[37] -1.811820215 -0.117047084 -0.698122450  0.152198274 -0.153932133 -0.344854070 -0.503635532  0.442124918 -0.998293019
[46]  1.219133756  1.178315011 -2.084715497  0.804313978 -1.326328611
> 
> 
> dput(vnum)
c(-0.895220889527685, 0.472985785738034, -1.15366853263124, 0.299671809636945, 
-0.507962145051303, 1.12660861411969, -1.96411398863887, -0.102483497414014, 
0.869147815091296, 0.12494983977689, 0.00707556874660856, 1.05039945202467, 
0.127889864073583, 0.49532309962209, -1.44266897772691, -0.617975264009241, 
1.14286764387227, -1.42352302483503, 0.426532831792985, -0.18695831493576, 
1.64884779946544, -0.460224242900372, 1.65432921872007, 0.740546165726594, 
-0.558588184596504, 2.87685004055853, 0.829542673396929, -0.409403713666942, 
0.226379411797594, 2.72290397151737, 0.664555431373864, 0.661642149516906, 
0.47919874642046, -0.188802783058076, 0.667357712376591, 0.642126932641534, 
-1.81182021542319, -0.117047083654143, -0.698122449877863, 0.152198273727932, 
-0.153932132867208, -0.344854070273716, -0.503635531971558, 0.44212491839201, 
-0.998293018734691, 1.21913375551535, 1.17831501135198, -2.08471549710629, 
0.804313977822178, -1.32632861087304)

EDIT: As suggested in the answers and comments, I checked "boxplot(vnum, 100*vnum)". It is really impressive:

enter image description here

$\endgroup$
1
  • 2
    $\begingroup$ The sample means are obviously different, but "very" compared to what? Hint: boxplot(vnum,100*vnum) $\endgroup$ Oct 1, 2014 at 8:33

3 Answers 3

10
$\begingroup$

The answer to your question lies in the variance. The sample mean differ by a factor of 100, yes, but what happens with the variance if you multiply a stochastic variable by a constant?

Lets look at this in more detail:

$$ Y = 100\cdot X $$ The sample mean is then $$ \hat\mu_Y = \frac{1}{n}\sum_{i=1}^nY_i = \frac{1}{n}\sum_{i=1}^n\left(100\cdot X_i\right) = \frac{100}{n}\sum_{i=1}^n X_i = 100\cdot\hat\mu_X. $$ And the sample variance $$ \hat\sigma^2_Y = \frac{1}{n-1}\sum_{i=1}^n\left(Y_i - \hat\mu_Y\right)^2 = \frac{1}{n-1}\sum_{i=1}^n\left(100\cdot X_i - 100\cdot\hat\mu_X\right)^2 = \\ = \frac{100^2}{n-1}\sum_{i=1}^n\left(X_i - \hat\mu_X\right)^2 = 100^2\cdot \hat\sigma^2_X $$

So we get, as previously stated, a mean that is increased by a factor of 100, but at the same time the variance increase 10000 fold!

Looking at the variance of the data you supplied you can see that this applies:

R> var(vnum)
[1] 1.13196
R> var(100*vnum)
[1] 11319.6

When doing a Welch t-test the variance is an important term as this is what normalizes the difference between the samples to follow the student-t distribution:

$$ t = \frac{\hat\mu_X - \hat\mu_Y}{\sqrt{\frac{\hat\sigma^2_X}{n_X} + \frac{\hat\sigma^2_Y}{n_Y}}} $$

Here you can see that the large difference in mean will be cancelled out by the large difference in variance.

However, as the sample size increase the denominator will decrease and the difference will be significant. As you can see in this example:

R> set.seed(1)
R> vnum <- rnorm(10000, 0.1, 1)
R> t.test(vnum, 100*vnum)

    Welch Two Sample t-test

data:  vnum and 100 * vnum
t = -9.1394, df = 10001, p-value < 2.2e-16
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -11.23735  -7.26831
sample estimates:
mean of x mean of y 
 0.093463  9.346296 

I hope this answers your question.

Btw, looking at the boxplot as @Scortchi said can be helpful to get a feel for the distributions you are working with. There you can clearly see the increase in variance as well even if you will not see if the difference in the means is significant or not.

$\endgroup$
1
  • $\begingroup$ Thanks for explaining very well. I have added boxplots in the edit to my question above. $\endgroup$
    – rnso
    Oct 1, 2014 at 10:21
6
$\begingroup$

We can see pretty much everything we need from the empirical cdf of the two samples:

$\quad\quad$enter image description here

Relatively speaking, there's essentially no variation in vnum, so it's almost the same as doing a one sample test of 100vum where the hypothesized mean is the sample mean of vnum. And we can see that the red almost-line is pretty close to the middle of the wider sample - not far enough away that we could tell them apart unless the sample size was very large.

And both those thoughts turn out to be the case:

  Welch t-test:      t = 0.7637, df = 49.01, p-value = 0.4487
  One sample t-test: t = 0.7637, df = 49, p-value = 0.4487
$\endgroup$
-1
$\begingroup$

This is because neither mean(x) nor mean(y) is statistically different from 0. So, it is quite possible that really mean(x)=mean(y)=0, and the values 0.11 and 11 are due to statistical errors.

$\endgroup$
6
  • 2
    $\begingroup$ Only the difference between the means is being tested, so difference from zero is irrelevant. Otherwise, all right, but the answer's not really providing any intuition helpful for this particular case. $\endgroup$ Oct 1, 2014 at 8:51
  • $\begingroup$ Actually, it is relevant. Let the means be X and Y. If X=0 and Y=0 (which is possible) then X-Y=0 too. That is why the test gave you such a result. $\endgroup$
    – user31264
    Oct 1, 2014 at 15:10
  • $\begingroup$ One problem with the reasoning in this answer is that it is possible (and often occurs) that the mean of data $(x)$ is not significantly different from $0$, the mean of data $(y)$ is not significantly different from $0$, yet the difference of means can still be significantly different from $0$. Although, to make a point, @Scortchi may have exaggerated a little (but only a little) in asserting a comparison to $0$ is "irrelevant," it is therefore clear that a comparison to $0$ is more of a distraction than an illumination. $\endgroup$
    – whuber
    Oct 1, 2014 at 15:41
  • $\begingroup$ I wrote "X=0 and Y=0". If mean(y) is not significantly different from 0, but mean(x)-mean(y) is significantly different from 0, then both "mean(x)=0" and "mean(y)=0" are both above the threshold of significance but "mean(x)=0 and mean(y)=0" is not. $\endgroup$
    – user31264
    Oct 1, 2014 at 16:12
  • 1
    $\begingroup$ If one mean is -2 and other is +2, both may not be significantly different from 0 but could be significanltly different from each other. $\endgroup$
    – rnso
    Oct 1, 2014 at 16:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.