1
$\begingroup$

I'd like to apply a t-student test to compare diferent samples to reference one using R but first I'm checking if my data meets the requirements for t-test so I removed outliers using the function rm.outlier() from Rcmdr R package and performed a Shapiro-Wilk test to check if replicates and the reference data are normaly distributed and now I should perform a test to check homogeneity of variance comparing each sample with the reference.

The samples has 3 replicates each one while reference has 11 replicates and samples and reference are independent.

I though about a Fisher Test but R says I need the same amount of replicates in the sample and reference. Then I though about Levene's Test and I tryed to run it in R using the built-int function in Rcmdr package according to this post http://stats.stackexchange.com/questions/262026/most-suitable-test-to-check-homogeneity-of-variances

On the other side, I'm performing a Wilcoxon test which does not need so many assuptions and at the end compare the "possitives" from t-student and wilcoxon. Or if one sample or reference does not meet one of the t.test ssumtions use de results provided by wilcoxon test.

What would be your suggestion?

Improve this question
$\endgroup$
7
  • 1
    $\begingroup$ If I understand correctly you are performing a t-test to compare a group with 3 cases to a group with 11 cases? With a sample-size that low any test of your distributional assumptions is in vain. You should use a non-parametric test making no such assumptions (Mann-Whitney-U, boot-strapped t-test). $\endgroup$
    – mzunhammer
    Commented Feb 15, 2017 at 9:26
  • $\begingroup$ yes, I edited a bit the post explaining that I'm running a Wilcoxon test as well and im star to think that may I should use WIlcoxon and forget about t.test and all their assumtions. Would wilcoxon be valid? $\endgroup$
    – Neuls
    Commented Feb 15, 2017 at 9:27
  • 1
    $\begingroup$ Also, given the very small sample-size any data-driven "outlier exclusion" is likely to be flawed and have a huge influence on the outcome. Avoid at all cost. $\endgroup$
    – mzunhammer
    Commented Feb 15, 2017 at 9:30
  • $\begingroup$ Uhm i suspected so about outliers because when I performed rm.outlier in the samples with 3 replicas each one some values got removed... However, outlier points appear in reference which has 11 replicates when doin a boxplot of my data. So I decided only removing outlier in reference $\endgroup$
    – Neuls
    Commented Feb 15, 2017 at 9:33
  • 1
    $\begingroup$ I do not know which procedure the "rm.outlier() from Rcmdr R package" is using and at first glance the CRAN manual is providing very little information on this. But regardless of which method is used: with so few data-points it is hard to justify any data-driven outlier exclusion. I would use all data-points and a non-parametric test. $\endgroup$
    – mzunhammer
    Commented Feb 15, 2017 at 9:58

1 Answer 1

Reset to default
0
$\begingroup$

You are planning to perform a t-test to compare mean differences of two groups, one with with 3 cases, one with 11 cases. You are aware that there are assumptions that have to be met (sample must be drawn from a normally distributed population homogenous in variance) for the t-test to be valid. You are unsure whether and how you can safeguard that these assumptions are met. Further, you performed some type of automated outlier-exclusion, which is likely to be based on further assumptions about the distribution of your data.

In short, given your very low sample size you cannot check these your assumptions reliably. Given your small sample size, the outlier exlusion procedure will have a tremendous impact on your results.

The solution to your problem: Stick to a test that does not make distributional assumptions (Mann-Whitney-U) and do not exclude outliers by data-driven methods.

Although t-tests at very low sample sizes may be valid if all assumptions are met, there is no way you can check this with the data at hand and therefore it is better to default to a non-parametric test, if you do not have a good (external) source showing that your samples is taken from a normal distribution with homogenous variances.

For an excellent discussion of the pro's and con's of testing normality see: Rochon, Gondan, & Kieser M, 2012.

Improve this answer
$\endgroup$
2
  • $\begingroup$ So according to you explanation, I should use data retrieven from Wilcoxon test since its non parametric? hat was nice you included a citation I will read it carefully to fully understand this kind of problematics when performing some statistical tests! $\endgroup$
    – Neuls
    Commented Feb 15, 2017 at 12:48
  • $\begingroup$ Hold on i just found out Mann-withney is the same than Wilcoxon.. $\endgroup$
    – Neuls
    Commented Feb 15, 2017 at 13:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.