1
$\begingroup$

I am working with a dataset with only 8 subjects. I am working with a rare event so increasing the sample size is not a possibility. I was wondering if I can do a one sample T-test with this data if the normality assumption is satisfied? Or a Wilcoxon one sample signed rank test, if not? In addition, does it make sense to do a test (parametric or nonparametric) with only 8 subjects? Thank you in advance for your response!

$\endgroup$
  • 1
    $\begingroup$ What null or working hypothesis are you considering? $\endgroup$ – Michael M Apr 25 '17 at 17:15
  • 4
    $\begingroup$ @Mark That is categorically false. There are plenty of things that can be definitively established, beyond a reasonable doubt, with much smaller sample sizes than 8. $\endgroup$ – whuber Apr 25 '17 at 17:26
  • 1
    $\begingroup$ @Mark whuber is right. Statistical tests can work in small samples if the evidence in the data is strong enough. $\endgroup$ – Michael R. Chernick Apr 25 '17 at 17:56
  • 1
    $\begingroup$ @Mark: Like you measure IQs for $n=8$ randomly chosen patients from an institution. You get scores 81, 82, 83, 84, 85, 86, 87, and 88. You do a t-test and can be very certain ($P \sim 10^{-7}$)that patents in this institution have lower than average IQs. Definitely agree with whuber here. $\endgroup$ – David Wright Apr 25 '17 at 17:58
  • 1
    $\begingroup$ @Mark re "like what?": "Student," in his original 1908 paper, applied his test to samples of size four. As I recall, they were anthropometric data. He probably really had in mind quality control data for beer :-). $\endgroup$ – whuber Apr 26 '17 at 22:07
3
$\begingroup$

Of course you can do a test with 8 observations.

The reason is that tests of significance were designed and developed for small samples. Before that, scientists had to collect very large samples so they could look at the descriptive statistics and histograms and say things like "Hmm, looks pretty different to me." Then their friend(?) down the hall would say something like "Hmmm, I'm not sure. Looks like some funny outliers there."

Tests of significance permitted scientists to come to conclusions with small samples.

This was Student's (true name, WS Gossett) problem as a chemist for Guinness Breweries. He took samples of stuff, tested it in the lab, and then did statistical comparisons. Doing lab work used up his time and he could often only get small samples.

Some samples are too small, of course. I think five is about as low as you can go because (1/2)^5 is 1/32 or just smaller than .05 but (1/2)^4 is 1/16 or just larger than .05. But then .05 isn't so important anyway so maybe n = 4 is about as low as you can go (since (1/2)^3 = 1/8) and still use statistics.

$\endgroup$
  • 1
    $\begingroup$ A sample of one is as low as you can go and still use statistics. Although considerations like $1/2^5$ are indeed useful guideposts, it's important not to confuse them with the actual information you have about the sampling distribution of the mean. $\endgroup$ – whuber Apr 26 '17 at 22:09
  • $\begingroup$ That, of course, would be the situation that calls for use of the word "datum" instead of "data." $\endgroup$ – David Smith Apr 26 '17 at 22:35
  • $\begingroup$ Thanks, David and whuber for your useful comments and answers. $\endgroup$ – curiousmind May 1 '17 at 21:27
1
$\begingroup$

Likert scale data is problematic for a lot of standard statistical techniques, but in your case the sample size is probably too small to resolve the discreteness of your scale. It's probably okay to do a t-test, but I would probably choose the more non-parametric sign test.

$\endgroup$
  • $\begingroup$ Someone must have done a simulation on this but I don't know of one. I believe simulations way back in the 1950's showed ANOVA on dichotomous data had fairly accurate p values. I don't remember but I assume this would not be true with extremely different proportions of 1's and 0's. $\endgroup$ – David Lane Apr 25 '17 at 18:35
  • $\begingroup$ Thank, David. What I can also do I assume is a sensitivity analysis. Do the t-test if the normality assumption satisfies, as well as the non-parametric, to see if the conclusion obtained is similar. $\endgroup$ – curiousmind Apr 25 '17 at 19:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.