# Can a paired t-test be used with the number of words recalled?

I recently went to a 3rd grade level science/math school exhibition, where someone had posted this data on how well people remember words, when they read it in a book vs a tablet. There were in total 10 similar (if not same) words which the participants were asked to recall from two paragraphs (same participants on book vs tablet, but different paragraphs). (For the sake of discussion assume that the data is collected scientifically)

tablet: 8  8  8  6 10  6  4 10 10  6  4  6 10  9 10  9  9 10  9  7
book: 10 10 10  9  9 10  9 10 10  6  4  8  9 10 10 10 10 10  3  9


I have applied paired T test [( mean of diff - 0) / std dev of diff/sqrt(n=20) ] to figure out if the difference is statistically significant at 95% confidence level. I used two tailed T test and came up with the fact that this difference is not significant since test statistic is 1.686 which is well with in the range of T values for .975 with 19 degrees of freedom (+- 2.093024)

So my questions are:

• Is my assumption correct to use paired T-test?
• If not then what is the right test?
• and if and only if possible is my calculation correct?

I also have another query:

Where can I get a book where I can get tons of question on stat, but where I don't know the context of the question in which it is being asked. Like right now when I'm solving problems from a book I know that I'm reading chapter on paired t-test so I've to apply this solution.

• You can't use the t-test for these data because they are number correct out of a fixed total (binomial data). – gung - Reinstate Monica Jan 26 '16 at 4:35
• @gung is right. The t-test is not appropriate here. You might consider using McNemar's test. For the book, try Problem Solvers on statistics. It's just a bunch of random stats questions without any context given.: amazon.com/Statistics-Problem-Solver-Solvers-Solution/dp/… – StatsStudent Jan 26 '16 at 4:54

Each data point (e.g., tablet: 8) is actually the number remembered correctly out of a finite total possible words (10). Numbers like these are binomial. They cannot be normal, and although they might be 'normal enough' in some circumstances, they won't be in yours.

A common analysis with data like these would be to run a logistic regression. It is possible to do that with paired data, but it requires some fancier machinery. If you're new to statistics, you might appreciate something simpler.

A nonparametric test like the Wilcoxon signed rank test might be appropriate. In essence, you are testing if one number tends to be higher than the other, but ignoring by how much. You are only using the ordinal information in the data.

Here is an example, coded in R:

d = read.table(text="8  8  8  6 10  6  4 10 10  6  4  6 10  9 10  9  9 10  9  7
10 10 10  9  9 10  9 10 10  6  4  8  9 10 10 10 10 10  3  9")

d = as.data.frame(t(as.matrix(d)))
names(d) = c("tablet", "book")
rownames(d) = NULL
with(d, wilcox.test(tablet, book, paired=TRUE))
#  Wilcoxon signed rank test with continuity correction
#
# data:  tablet and book
# V = 20, p-value = 0.04246
# alternative hypothesis: true location shift is not equal to 0
#
# Warning messages:
#   1: In wilcox.test.default(tablet, book, paired = TRUE) :
#   cannot compute exact p-value with ties
# 2: In wilcox.test.default(tablet, book, paired = TRUE) :
#   cannot compute exact p-value with zeroes


The results are significant, but it complains about the ties in the data. If that bothers you, you could try a permutation test. The idea is just to shuffle the rows independently of each other on every iteration for lots of iterations to determine the null distribution of the Wilcoxon signed rank test statistic. Then we can compare your observed test statistic to that distribution.

d2     = as.matrix(d)
v.vect = vector(length=1000)
nd     = matrix(NA, nrow=20, ncol=2)
set.seed(7998)
for(i in 1:1000){
for(j in 1:20){
nd[j,] = sample(as.vector(d2[j,]), size=2)
}
v.vect[i] = wilcox.test(nd[,1], nd[,2], paired=TRUE)\$statistic
}
summary(v.vect)
# Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
# 6.00   40.00   52.00   51.84   63.00   94.00
2*mean(v.vect<20)
# [1] 0.04


It is still significant. In the original dataset, people remembered more words from the book than the tablet 55% of the time, more from the tablet 15% of the time, and equal from both 30% of the time. That is rather unlikely to occur when words from books and tablets are equally memorable.

I don't know about the book request. @StatsStudent's suggestion seems reasonable.

On the other hand, there may be some issues with this study. Namely, I gather the words and paragraphs differ between the conditions, which is a problem. It might just be that the words and paragraphs used in the book are more memorable. What the exhibitor could have done is have 10 people test with one set of words in the book and the other 10 on the tablet, and switch the words for the last 10 people. Then the words and the conditions would be counterbalanced, and would not be confounded. That makes it possible to make a claim about the conditions without worrying that the effect is really due to the words.

• Hi gung thanks so much for nicely explaining. So quick follow up question: what if we're to tell you that the data is score of 20 students taken in the morning and evening. (I guess then it will be a candidate for paired t test as this data is normally distributed and is numerical) – user3105943 Jan 27 '16 at 1:27
• @user3105943, the Wilcoxon signed rank test that I illustrate is for paired data. Your data are paired, so a paired test is appropriate. However, your data are not normal, & cannot be normal, by definition; they are binomial. There are other, more sophisticated, ways of dealing with that, but the Wilcoxon will be easier for you. – gung - Reinstate Monica Jan 27 '16 at 1:41