4
$\begingroup$

In his paper "Ten ironic rules for non-statistical reviewers", Karl Friston includes the following tongue-in-cheek response of a fictional author to a reviewer who complains about the sample size being too low:

“We suspect the reviewer is one of those scientists who would reject our report of a talking dog because our sample size equals one!”

But in all seriousness, how would one formally make the distinction between sample-to-population inference - with its rules relating sample size, power etc - and single-case observations that undeniably prove a conceptual point? Talking dogs aside, this point could be, for instance, a brain lesion patient who nonetheless is still able to perform a cognitive function for which the lesioned brain area used to be thought necessary for. Surely in this case the sample size N=1 would not prevent a strong claim being made from this neuropsychological result?

Improve this question
$\endgroup$
2
  • $\begingroup$ +1, good question, but I find your title confusing, too specific, and do not quite see how it is related to the body of your question (why neuropsychology? why cognitive neuroscience?). Perhaps simply "Reliability of single case reports" would be clearer? $\endgroup$
    – amoeba
    Commented Sep 18, 2016 at 11:57
  • $\begingroup$ you are right, i'll edit it $\endgroup$
    – z8080
    Commented Sep 18, 2016 at 13:39

1 Answer 1

Reset to default
2
$\begingroup$

Case studies are one of many examples of valid scientific methods that do not use statistics. While we can draw limited statistical conclusions from samples of size one, science not always uses statistics.

As far as I know, such methods are used for example in neurology where case reports of rare brain injuries has helped us to learn about human brain. In many such cases experimental studies would not be possible for technical and ethical reasons and large-sample observational studies are impossible since prevalence of such injuries is too small.

Notice that on grounds of logic it is often enough to have single case to disprove something (e.g. you need single flying machine to disprove that "heavier-than-air flying machines are impossible").

Moreover, in some disciplines of science (I have in mind mostly social sciences) observational studies are the only possible, imagine for example cultural anthropologist observing some small tribe in the middle of amazonian forest. In such cases you can make limited use of statistics, but you would mainly depend on non-statistical methods.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks for this helpful answer. It is your 3rd paragraph that my question alluded to, and which best answers it, i.e. the report of N=1 talking dogs is enough to disprove the assumption "there are no talking dogs". It's still interesting to note that hypothesis tests, that require sample sizes of N>1, have nevertheless the same aim of disproving (falsifying) something - namely, the null hypothesis. $\endgroup$
    – z8080
    Commented Sep 18, 2016 at 11:37
  • $\begingroup$ Putting aside logistics/ethics of whether samples of N>1 can/should be collected, can we say that a falsification obtained with (say) N=20 is more reliable/believable than one with N=1. Would it make any sense to say that the former is a statistical falsification and the latter merely a conceptual one? $\endgroup$
    – z8080
    Commented Sep 18, 2016 at 11:37
  • 2
    $\begingroup$ @wildetudor The difference is that null hypotheses (that are rejected in statistics) make some probabilistic claims about how some properties are distributed in the population. Whereas hypotheses that can be disproved with N=1 are "absolute". Take "there are no talking dogs": the distribution of talking ability among dogs is hypothesized to be a delta function at zero, and so it takes N=1 to disprove it. In contrast, a null hypothesis "adults dogs mean weight is 10 kg" assumes a nontrivial distribution of weights across dogs; you need a large N to be able to say if the mean is 10 or not. $\endgroup$
    – amoeba
    Commented Sep 18, 2016 at 11:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.