# How do I bootstrap with R correctly without increasing the degree of freedom?

I need to compare survey responses from my experimental (referred as "FS" below) and control group (referred as "GV" below) using independent samples t-test. As the two groups have unequal sample size (n1=87,n2=64), I'm thinking about using bootstrapping to compensate for it, assuming our sample is representative of the population.

I will list responses of both groups to one survey question (Question 4, referred as Q4 below) as an example of how I did bootstrapping and t-test. All questions are asked to respond on a 7-point Likert scale.

## what I have done for bootstrapping on R
## FS (experimental group) has a total number of 87 responses, listed as below
Q4_FS <- c(2,3,3,3,3,4,4,4,4,4,4,4,4,4,4,4,4,5,5,5,5,5,5,5,5,5,5,6,6,6,
6,6,7,7,7,7,3,3,4,4,4,4,6,6,6,7,7,7,2,3,3,3,3,4,4,4,5,5,5,
5,5,6,6,6,7,7,7,2,2,3,3,3,3,3,3,3,3,4,4,4,5,5,5,6,6,6,6,6,7)

## to resample with replacement by bootstrapping
mQ4FS <- c()
for (i in 1:1000) {
Q4FS  <- sample(Q4_FS,87,replace=T)
mQ4FS <- c(mQ4FS,Q4FS)
}

## GV (control group) has a total number of 62 responses, listed as below
Q4_GV <- c(5,4,5,3,5,5,4,4,5,4,7,5,3,3,4,3,5,4,4,5,4,7,7,7,5,4,7,2,
3,2,3,2,3,4,5,6,5,3,5,6,4,5,7,4,2,6,7,6,6,4,3,5,5,6,3,4,4,3,6,6,3,3)

## resample with replacement
mQ4GV <- c()
for (i in 1:1000) {
Q4GV  <- sample(Q4_GV,62,replace=T)
mQ4GV <- c(mQ4GV,Q4GV)
}

## perform independent samples t-test
t.test(mQ4GV,mQ4FS)

## results of t-test
Welch Two Sample t-test

data:  mQ4GV and mQ4FS
t = -20.032, **df = 133890**, p-value < 2.2e-16
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
-0.1635008 -0.1343572
sample estimates:
mean of x mean of y
4.486887  4.635816


The unusually large degree of freedom indicates that by "resampling", I increased both samples from a size of 62 and 84 to 62000 and 84000. I am wondering how I could avoid this? Is there any other code I can use to bootstrap?

• @Dole IMHO this question has more statistic that programming issues in it, so it is on-topic. – Tim Nov 4 '15 at 20:18
• It's not clear to me why there's any need to "compensate" for a difference in sample size. The t-test already does that. – Glen_b Nov 4 '15 at 20:24
• Although this question asks for code, the OP's difficulties are due to misunderstandings about the nature of bootstrapping. I think this Q will be better addressed here (as a statistical question) than on Stack Overflow (as a coding question). – gung Nov 4 '15 at 20:41
• Thanks @gung. could you elaborate on "misunderstandings about the nature of bootstrapping"? – Ariana K. Nov 4 '15 at 21:27

Bootstrapping like that is giving the t-test the impression that there is more data than there actually is. With lots of data, even small differences are seemingly significant. You should try just running the t test:

Q4_FS <- c(2,3,3,3,3,4,4,4,4,4,4,4,4,4,4,4,4,5,5,5,5,5,5,5,5,5,5,6,6,6,
6,6,7,7,7,7,3,3,4,4,4,4,6,6,6,7,7,7,2,3,3,3,3,4,4,4,5,5,5,
5,5,6,6,6,7,7,7,2,2,3,3,3,3,3,3,3,3,4,4,4,5,5,5,6,6,6,6,6,7)

Q4_GV <- c(5,4,5,3,5,5,4,4,5,4,7,5,3,3,4,3,5,4,4,5,4,7,7,7,5,4,7,2,
3,2,3,2,3,4,5,6,5,3,5,6,4,5,7,4,2,6,7,6,6,4,3,5,5,6,3,4,4,3,6,6,3,3)

## perform independent samples t-test
t.test(Q4_GV,Q4_FS)


From Wikipedia:

Welch's t-test is more robust than Student's t-test and maintains type I error rates close to nominal for unequal variances and for unequal sample sizes.

• Thanks Andrew! I am using the same methods to compare the results of a voting activity. I just posted this question on stackexchange. Could you please share with me your thoughts? Here is the link: stats.stackexchange.com/questions/180258/… TONS OF THANKS! – Ariana K. Nov 5 '15 at 1:18