Verifying the assumptions of the paired t-test

Question

I have data on the number of cases that customers open (with an engineer) at a computer company each month, across the various technologies that are supported. The assumption is the documents are located on the company's support website / the customers actually view the documents before opening a case. These data include cases that were opened before and after any documentation was created. I am interested in studying the overall effect that the documentation has on case deflection. I am considering using the paired t-test but I have a few questions about verifying the assumptions, and about other reasonable tests to use if the assumptions aren't satisfied. For this example say there are 35 different technologies, and there are 200 supporting documents divided up unequally among the 35 technologies. Below is a an example to make my problem more clear, followed by my questions.

Example: Say there is a year's worth of data for technology $x_1,~x_2,~...,~x_{35}$ pre / post documentation creation. For instance, technology $x_1$ may have had 568 cases opened prior to documentation creation, then say we noticed that 276 cases were opened for $x_1$ after 46 documents were created for $x_1$. For $x_2$ there were 438 cases that were created prior to documentation creation, and 155 cases opened after 27 documents were created for $x_2$, etc. I want to study the overall effect that the documents as a whole had on case deflection.

I listed my questions as bullet points:

• Since one document is not being applied to all of the technologies can I still use the paired t-test? Does it make sense to use the paired t-test in this example?
• One assumption of the paired t-test is that the paired differences are normally distributed. What efficient / easy to use tests that I can utilize to check for normality? I can utilize both R and Excel for the check.
• Another assumption for the paired t-test is that the pairs are independent. How does independence apply to my situation?
• If the paired differences are not normally distributed, then what other test can I use for my problem? I came across the "Wilcoxon signed-rank test", would this be an appropriate test to use? If so, what assumptions must be met to use it / would it apply in my problem?

Answer 1

My first impression on reading your description would be to use a Generalized Linear Mixed Effects model (GLMM) with a random effect for technology and with the response was distributed as a Poisson. However, this may be overkill, and I gather, is more advanced than you would feel comfortable with.

For your situation, I would probably suggest you use the Wilcoxon signed rank test, as you had surmised. The assumptions are listed on the Wikipedia page; I see no reason to worry that it wouldn't be appropriate for your situation.

If you really wanted to try the paired-samples t-test, you could calculate a vector of differences and assess them for normality. My preference would be to make a qq-plot, but if you prefer a hypothesis test, you could use the Shapiro-Wilk test.

Regarding your other questions, I don't think you need to worry about the fact that different documents have been prepared for the different technologies. Lastly, the independence of the pairs means that there is nothing about the fact that one pair moved in a particular direction that influence the fact that another pair did.