I have data on the number of cases that customers open (with an engineer) at a computer company each month, across the various technologies that are supported. The assumption is the documents are located on the company's support website / the customers actually view the documents before opening a case. These data include cases that were opened before and after any documentation was created. I am interested in studying the overall effect that the documentation has on case deflection. I am considering using the paired t-test but I have a few questions about verifying the assumptions, and about other reasonable tests to use if the assumptions aren't satisfied. For this example say there are 35 different technologies, and there are 200 supporting documents divided up unequally among the 35 technologies. Below is a an example to make my problem more clear, followed by my questions.

Example: Say there is a year's worth of data for technology $x_1,~x_2,~...,~x_{35}$ pre / post documentation creation. For instance, technology $x_1$ may have had 568 cases opened prior to documentation creation, then say we noticed that 276 cases were opened for $x_1$ after 46 documents were created for $x_1$. For $x_2$ there were 438 cases that were created prior to documentation creation, and 155 cases opened after 27 documents were created for $x_2$, etc. I want to study the overall effect that the documents as a whole had on case deflection.

I listed my questions as bullet points:

  • Since one document is not being applied to all of the technologies can I still use the paired t-test? Does it make sense to use the paired t-test in this example?
  • One assumption of the paired t-test is that the paired differences are normally distributed. What efficient / easy to use tests that I can utilize to check for normality? I can utilize both R and Excel for the check.
  • Another assumption for the paired t-test is that the pairs are independent. How does independence apply to my situation?
  • If the paired differences are not normally distributed, then what other test can I use for my problem? I came across the "Wilcoxon signed-rank test", would this be an appropriate test to use? If so, what assumptions must be met to use it / would it apply in my problem?
  • 1
    $\begingroup$ It seems to me that you have counts, not the kind of continuous data for which the t-test (paired or otherwise) is appropriate. You have values for the number of cases that were opened in each situation, do you know the total number of cases that could possibly have been opened? If so, we can treat these as binomial, if not, we can treat them as Poisson. $\endgroup$ – gung - Reinstate Monica Jun 8 '13 at 17:48
  • $\begingroup$ I do not know the number of total cases that could have been possibly opened, this is due to the uncertainty of the customer behavior. I am not familiar at all with Poisson. Given this information what sort of test do you think would be appropriate for this problem? What additional information would we need? $\endgroup$ – Quaternary Jun 8 '13 at 18:03

My first impression on reading your description would be to use a Generalized Linear Mixed Effects model (GLMM) with a random effect for technology and with the response was distributed as a Poisson. However, this may be overkill, and I gather, is more advanced than you would feel comfortable with.

For your situation, I would probably suggest you use the Wilcoxon signed rank test, as you had surmised. The assumptions are listed on the Wikipedia page; I see no reason to worry that it wouldn't be appropriate for your situation.

If you really wanted to try the paired-samples t-test, you could calculate a vector of differences and assess them for normality. My preference would be to make a qq-plot, but if you prefer a hypothesis test, you could use the Shapiro-Wilk test.

Regarding your other questions, I don't think you need to worry about the fact that different documents have been prepared for the different technologies. Lastly, the independence of the pairs means that there is nothing about the fact that one pair moved in a particular direction that influence the fact that another pair did.

| cite | improve this answer | |
  • $\begingroup$ Completely pedantic side note: "data" is plural (the singular is "datum"), thus "these data include" is grammatical, whereas "this data includes" is not. $\endgroup$ – gung - Reinstate Monica Jun 9 '13 at 19:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.