I have data from an experiment where I applied two different treatments in identical initial conditions, producing an integer between 0 and 500 in each case as the outcome. I want to use a paired t-test to determine whether the effects produced by the two treatments are significantly different. The outcomes for each treatment group are normally distributed, but the difference between each pair is not normally distributed (asymmetric + one long tail).

Can I use a paired t-test in this case, or is the assumption of normality violated, meaning I should use a non-parametric test of some kind?

  • $\begingroup$ The experiment is based around a simulation. I can set the initial conditions of the simulation as I please. Thus, for each pair, I start with the same initial conditions, and apply two different algorithms. $\endgroup$ – John Doucette Sep 22 '11 at 20:25
  • $\begingroup$ From what you describe these sounds like independent groups. Did you apply both treatments to each case or is there some other matching? What is the correlation between the conditions? Your wording is odd... do you mean you have one value in the tail making it asymmetric? $\endgroup$ – John Sep 22 '11 at 20:30
  • $\begingroup$ Thinking about it further, I'm less certain that they are dependent, but perhaps you can shed some light on it. The analogous correlation in the real world would be: I have a person. Treatment one is administered, and a measurement is taken. Then I roll back time, and instead administer treatment two. A measurement is again taken. It seems to me that these measures should be considered correlated. Perhaps they should not? $\endgroup$ – John Doucette Sep 22 '11 at 20:39
  • $\begingroup$ Also, with the non-normality, the distribution is both asymmetric and has one long tail (with multiple outliers). Removing some of the outliers would not make it normal. $\endgroup$ – John Doucette Sep 22 '11 at 20:40
  • 4
    $\begingroup$ If the univariate distributions are Normal and independent, then the distribution of differences must be Normal. Its lack of Normality demonstrates a dependence between the two distributions. The dependence is not merely that of correlation: there must be something else going on, too. $\endgroup$ – whuber Sep 22 '11 at 20:50

A paired t test only analyzes the list of paired differences, and assumes that sample of values is randomly sampled from a Gaussian population. If that assumption is grossly violated, the paired t test is not valid. The distribution from which the before and after values are samples is irrelevant -- only the population the differences are sampled from matters.

  • $\begingroup$ So let's say if I analyzed a non-linear model and generated y_observed at time = i. Can I do a paired t- test that compares each observed to actual value at time i? Let's also assume that I have the observed data for 100 time intervals, and forecast my numbers to the same intervals $\endgroup$ – dassouki Oct 19 '11 at 18:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.