2
$\begingroup$

I have a cohort of subjects (n=262) with a single-time intervention. Subjects have been asked to answer on a questionnaire before and after the intervention. The questionnaire comprises five items where each item is in the range [0, 1, 2, 3] and the total score is the mean value of these 5 items. Thus, the total score of each subjects is given by a rational number in the range 0-3.

Here is the histogram of their responses:

enter image description here

I applied a standard paired t-test to find an affect of the intervention on the cohort and found (with python stats):

>> scipy.stats.ttest_rel(data_before, data_after)    
>>Ttest_relResult(statistic=3.4864105747384686, pvalue=0.00074914757737233801)

So, it looks interesting and supports nicely my hypothesis that the intervention has affected the cohort. But I am wondering if the application of paired t-test on these skewed data with n=262 is justified in this case?

Any suggestion how to check my findings with other tests?

UPDATE

I shared the data through dropbox

$\endgroup$
9
  • 2
    $\begingroup$ Did you try the bootstrap? Did you try log transform or boxcox transform? Wilcox <or other nonparametric test? $\endgroup$ Oct 17, 2016 at 11:00
  • 4
    $\begingroup$ Paired t-tests make assumptions about the differences, not about the raw data. $\endgroup$
    – Peter Flom
    Oct 17, 2016 at 11:04
  • $\begingroup$ @PeterFlom, Sure, according to (statisticssolutions.com/manova-analysis-paired-sample-t-test) "Paired sample t-test is a statistical technique that is used to compare two population means in the case of two samples that are correlated. Paired sample t-test is used in ‘before-after’ studies, or when the samples are the matched pairs, or when it is a case-control study" So, this is exactly what I have in my setup. The same population before and after the intervention, so theoretically the application of the paired t-test is justify. Please correct me of I'm wrong. $\endgroup$ Oct 17, 2016 at 11:12
  • 1
    $\begingroup$ Although hard to tell from the plot the differences are at least unimodal and approximately symmetric so you may be OK with your t-test. $\endgroup$
    – mdewey
    Oct 17, 2016 at 12:34
  • 1
    $\begingroup$ The point made by @PeterFlom is that the only distribution relevant to this t-test is the difference distribution, which is not (appreciably) skewed. It is, however, leptokurtic, suggesting that the p-value cannot wholly be trusted. In particular, the histogram masks some of the most important statistics: exactly how many of the differences are exactly zero? How many are positive and how many are negative? $\endgroup$
    – whuber
    Oct 17, 2016 at 15:17

1 Answer 1

3
$\begingroup$

The original values aren't assumed to be normal, the differences are, so the skewness in the first two histograms is not an issue.

While your differences aren't normal, they're bounded, relatively symmetric and not very heavy-tailed (somewhat fat, with peaky center, but the boundedness helps), so this may not affect the t-test much.

The main concern would be that it looks like there might be a lot of 0 values, but a quick simulation with numbers very similar to yours seems to indicate very little problem with the distribution of the usual one sample t-statistic -- i.e. the significance level should be very close to the chosen level.

Power might be mildly affected by the heavier tails, but I wouldn't have much concern in this case.

I really don't see that there would be much problem here.

If you are concerned about challenges on the t-test, you could always consider a permutation test of the mean differences. [An alternative might be a Wilcoxon signed rank test but the high proportion of ties could be a concern.]

$\endgroup$
2
  • $\begingroup$ thanks for your answer. Indeed, there are quite a lot of zeros. Are there statistical tests which are suitable in the case of leptokurtic distribution? $\endgroup$ Oct 18, 2016 at 12:41
  • $\begingroup$ If your interest is primarily on identifying differences in the mean, the permutation test already mentioned in the answer would be my first thought. The discreteness (especially the high proportion of 0's) is the main point of concern, I think; studies that look at what might be suitable for heavy tails generally don't consider that particular issue $\endgroup$
    – Glen_b
    Oct 19, 2016 at 1:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.