0
$\begingroup$

I'm new to R and statistical data analysis in general.

Here is what I am trying to do:

I have data from patients before and after a therapy. I want to compare the means of the two measures to see if there is a significant difference to see if the therapy had any effect.

First I checked whether the data is normal distributed by doing a shapiro-wilk test:

differences <- data$SymptomsBefore - data$SymptomsAfter       #calculate the differences
shapiro.test(differences)                                     #do the test 

Output:

Shapiro-Wilk normality test

data:  differences
W = 0.92445, p-value = 0.2878

--> Since the p-value is bigger than 0.05, I can assume that the data is normal distributed.

So now I can do a paired samples t-test. Since I don't know if the effect is positive or negative I choose the two-tailed option.

t.test(data$SymptomsBefore, data$SymptomsAfter, paired = TRUE, alternative = "two")

Here is the result:

    Paired t-test

data:  data$SymptomsBefore and data$SymptomsAfter

t = -2.8939, df = 12, p-value = 0.01348

alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -1.5506404 -0.2185903

sample estimates:
mean of the differences 
             -0.8846154 

So, if I interpret the results correctly, this means there was a significant difference in the means (because the p-value was less than 0.05), meaning the therapy had an effect.
Also, on average the symptoms were -0.8846154 lower than before the therapy.

In a paper I would report that like this: "The results indicate that the therapy resulted in an decrease in Symptoms, t(12) = -2.8939, p = .01348."

Is my interpretation of the R-output correct? Am I missing something? Is there something I still need to check that I didn't think of?

Thank you for your help.

$\endgroup$
5
  • 1
    $\begingroup$ --> Since the p-value is bigger than 0.05, I can assume that the data is normal distributed. ... this is false. Since the p-value is bigger than 0.05, you can not reject the H0. (... and H0 is normality) $\endgroup$
    – jogo
    Jun 20, 2020 at 20:16
  • 1
    $\begingroup$ 12 degrees of freedom, so an N of 13 in each group? That's not enough for your Shapiro test to tell you anything. (Nobody can tell whether data is normal from 12 samples.) Run a non-parametric test, e.g. wilcox.test(). $\endgroup$
    – dash2
    Jun 20, 2020 at 21:01
  • $\begingroup$ Also, without a control group, you cannot ascribe the difference to the therapy. All you can say is that there’s was a difference. (The patients might have improved even if they hadn’t received the therapy.) $\endgroup$
    – Limey
    Jun 20, 2020 at 21:26
  • $\begingroup$ @dash2 How powerful would you expect wilcox.test to be with so few observations? $\endgroup$
    – Dave
    Jun 20, 2020 at 22:16
  • $\begingroup$ Not very powerful! But the t test is only going to be more powerful because it makes assumptions that cannot be validated (and if SymptomsBefore is a count, are very unlikely to be true....) For example, here's what happens if you run shapiro tests with very non-normal data: tmp <- replicate(1000, {x <- runif(13); shapiro.test(x)$p.value}) ; table(tmp < 0.05). I get power of about 10% to reject normality. $\endgroup$
    – dash2
    Jun 22, 2020 at 10:18

1 Answer 1

4
$\begingroup$

I will consider your Question along with several of the accumulated Comments:

Are data normal? A Shapiro-Wilk test on a sample of size 13 can give you some idea whether the population from which the sample was chosen was uniform.

Below we look at 100,000 simulated samples of size 13, from each of standard uniform, exponential, and normal populations.

  • As is to be expected, the Shapiro-Wilk test at the 5% level did reject about 5% of truly normal samples as not consistent with normal data.
  • However, only about 10% of samples from a uniform population (mainly symmetrical and almost never with outliers) were rejected as not normal, and
  • About 59% of samples from an exponential population (highly skewed and typically with outliers) were rejected as not normal.

The S-W test is one of the best tests of normality, but hardly definitive for samples as small as yours.

set.seed(2020)
pv.u = replicate(10^5, shapiro.test(runif(13))$p.val )
mean(pv.u <= .05)
[1] 0.10436

pv.e = replicate(10^5, shapiro.test(rexp(13))$p.val )
mean(pv.e <= .05)
[1} 0.59062


pv.n = replicate(10^5, shapiro.test(rnorm(13))$p.val )
mean(pv.n <= .05)
[1] 0.04858

Paired t test or Wilcoxon signed rank test? Moreover, it is not the individual observations that need to be nearly normal in order to get reliable results from a paired t test. It is the sample mean that needs to be nearly normal. If a sample of about a dozen differences is roughly symmetrical and without extreme outliers, then one can usually rely on a paired t test to give useful results.

I have only your results from a paired t test. Without your data, I can't look to see for sure what results you would have gotten from a Wilcoxon signed rank test on the differences. Especially with such a small sample, the Wilcoxon test is less powerful (likely to reject when $H_0$ untrue) than a t test.

Working backward from your printout it, I deduce that you must have had $\bar D \approx 0.88$ with standard deviations $S_D \approx 1.1.$ Symmetrical data centered at 0.88 and with SD 1.1 would lead the Wilcoxon test to reject with probability about 73%. So using a Wilcoxon test might not have been a bad choice. Without seeing your actual data, I can only guess.

pv.wilcox = replicate(10^5, wilcox.test(rnorm(13, .88, 1.1))$p.val)
mean(pv.wilcox <= .05)
[1] 0.73003

Your summary. In your summary you can only claim that patients reported lower symptom scores after the period of treatment than before. The improvement might have been either from the treatment or the passage of time. I think it is reasonable to give the mean of difference, the SD of differences, and the number of differences in treatment scores along with the P-value.

Whether this is for a local tech report or to be submitted to a journal for publication, an editor may ask you for a slightly different summary, perhaps including Cohen's D (which can be obtained from sample size, mean and SD). If you feel that a decrease of about 1 (0.88) on your symptom scale amounts to an important improvement for patients, then somewhere in your report you need to explain your scale and say why the average observed difference is of practical importance. If you have no control group, you must say so explicitly.

Note: If you have any data on decrease in symptoms for untreated subjects, it would be useful to mention that, commenting on similarities and differences (demographic and seriousness of condition) of those patients from your treated ones.

$\endgroup$
3
  • 1
    $\begingroup$ There's a small error in the code. You have mean(pv.u < 0.5) instead of mean(pv.u < 0.05). So you overestimate the Shapiro test's power to reject normality at 5% significance. $\endgroup$
    – dash2
    Jun 22, 2020 at 10:22
  • 1
    $\begingroup$ You might want to mention that the test on runif now only rejects 10% of the time not 74% :-) $\endgroup$
    – dash2
    Jun 23, 2020 at 18:26
  • $\begingroup$ @dash2: Thanks for letting me know about the errors. I hope I have fixed everything now. $\endgroup$
    – BruceET
    Jun 23, 2020 at 18:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.