# Statistical significance with sample size of 5 or less

Should small sample sizes of 5 and less be included in reports? This is an area of medical training that is currently being reported only to the course level trainers and statistically not included in reports to upper management. All of the data is mined from course-level surveys.

-
You should give some details about the test you want to do. This would give more adapted and focused answers. –  robin girard Dec 13 '10 at 18:48

If you are doing things properly the worst thing you can do is not to reject $H_0$ (does not mean $H_0$ is true, only mean you do not have enough evidence to reject)

Anyway, there can be something significant with 5 observations: If you need to test, for example, wether your random variable has a negative mean under the assumption of variance one, then having 5 observation over 10 is highly significant ....

Just be carefull not to use a test that uses something like asymptotic normality or other assumption that cannot be satisfyed or checked with only five observations

-

It depends what you're testing.

If you are testing for variance and your $H_0$ is that $V = 1$, but your values are $0, +1000, -1000, +2000, -2000$, then it's clear that you should reject $H_0$. If you are testing the hypothesis that the mean is not 2, then you do not have enough data to reject it.

For most things, though, 5 is too little.

(I didn't run through the math above to build intuition. I'd generally recommend a Bayes ratio as the best quantitative test).

-