How to interpret results on different t-tests for the same samples?

Question

I am performing a t-test on the same couple of vectors in R. The first vector is the list of daily returns on a stock. The second vector is the list of daily log - returns on the same stock. Now, I understand the assumptions why we do a paired t-test as opposed to a Welch t-test (the default t-test setting in R). These are following results:

Welch t-test

t.test(dailyreturns, logReturn)

Welch Two Sample t-test

data: dailyreturns and logReturn

t = 0.10813, df = 2350, p-value = 0.9139

alternative hypothesis: true difference in means is not equal to 0

95 percent confidence interval:

-0.0006801156 0.0007595007

sample estimates:

mean of x mean of y

0.0005027479 0.0004630553

Paired t-test

t.test(dailyreturns, logReturn, paired = TRUE)

Paired t-test

data: dailyreturns and logReturn

t = 15.866, df = 1175, p-value < 2.2e-16

alternative hypothesis: true difference in means is not equal to 0

95 percent confidence interval:

3.478409e-05 4.460108e-05

sample estimates:

mean of the differences

      3.969258e-05 

My question is given that the alternate hypothesis in both tests is the same and as the author mentions (see answer to Problem 14), while the assumptions in the Welch t-test is incorrect, the results are same. However, given the vastly p-values in both test, the null-hypothesis in one test will be rejected in the Welch t-test but won't be rejected in the paired t-test! And the graph in Problem 12 (from the same data set) shows that there isn't much difference in daily returns and daily log-returns. Should it not mean that the p-value should show insignificant results and therefore we will fail to reject the null hypothesis that the true difference in means is equal to 0? (Here, I am assuming true means ~ 0 and not EXACTLY 0 - am I right in this assumption?).

Answer 1

Ok, so comparing y and log(y) in a t test essentially amounts to comparing one sample against itself. Instead of taking two measurements, they've transformed one measurement and called it different. It would make no sense in real life, but i guess it makes sense for the purposes of an exercise.

The Welch T test is a special kind of two sample T test that does not assume equality of variance between the groups. However, it is still a two sample T test. Hence, it does not match the current situation.

The paired T test is essentially a one sample T test. Both "samples'" standard deviation are the same, because they are actually the same sample/group. So you don't have to pool their standard deviations to get standard error, so they're going to have smaller standard errors. A T test is always the difference of means divided by standard error. So if you have a bigger standard error on the bottom, you'll have a smaller T test.

This is exactly what happened in this case. The numerator is the same in both tests. But the denominator is much larger for the Welch T test. So it resulted in the Welch T test being much smaller and hence non significant. You can substitute toy numbers into the tests' equations below. The standard error of the Welch is always bigger.

Paired T test $$\frac{\bar{X_1}-\bar{X_2}}{\frac{S}{\sqrt{n}}}$$

Welch T test $$\frac{\bar{X_1}-\bar{X_2}}{\sqrt{\frac{S_1^2}{n}+\frac{S_2^2}{n}}}$$

Does that mean you always use the paired T test? Obviously NOT. You can't just reduce the standard error because you want to. It has to come from the data, and if the groups aren't the same, then you have to pool their standard deviations to get standard error.

In your case, this situation calls for going with the paired T test and hence, rejecting your null hypothesis. But I stress this is a bad example. A better one is if you tested the same people across two conditions and recorded their scores and wanted to see if the two conditions had different means. Or recording people's weights before and after dieting to see if, as a group, they've lost weight.