Return to Answer

Note: the above probably misses a few details in terms of control groups, side effects, etc.

Note: the above probably misses a few details about control groups, side effects, hypothesis testing, etc. It's just meant to give a basic idea of what the numbers might look like.

Now a one-line conclusion of a study could be "the drug likely cures some percentage of people" or "we don't whether it cures anyone".

A 10000-person study is going to end up saying "the drug likely cures some percentage of people" more often, even if the percentage is really tiny. A 10-person study will end up saying "we don't whether it cures anyone" more often.

When a 10000-person study says "we don't whether it cures anyone", we can be pretty sure that it cures between 0% and a very, very tiny percentage of the population. Whereas with a 10-person study with the same conclusion it could still cure a fairly large percentage. We just don't know yet.

Now a one-line conclusion of a study could be "the drug likely cures some percentage of people" or "we don't know whether it cures anyone".

A 10000-person study is going to end up saying "the drug likely cures some percentage of people" more often, even if the percentage is really tiny. A 10-person study will end up saying "we don't know whether it cures anyone" more often.

When a 10000-person study says "we don't know whether it cures anyone", we can be pretty sure that it cures between 0% and a very, very tiny percentage of the population. Whereas with a 10-person study with the same conclusion it could still cure a fairly large percentage. We just don't know yet.

A small sample size needs a more significant result if you want to draw a conclusion from it.

• If your drug cures 30% of 10 people, the percentage of the general population cured could be anywhere between like 0% or 60% of people.

• If your drug cures 30% of 10000 people, you can be quite sure it actually cures around 30% of people (more specifically, between 29% and 31% of people).

• If your drug cures 100% of 10 people, you can be quite sure it would cure around at least 60% of the general population.

• If your drug cures 100% of 10000 people, you can be quite sure it actually cures around 100% of people.

When a 10000-person study says "we don't whether it cures anyone", we can be pretty sure that it cures between 0% and a very, very tiny percentage of the population. Whereas with a 10-person study it could still cure a fairly large percentage. We just don't know yet.

Note: this answer uses "significant" to mean "practically significant" not "statistically significant". As in "something that actually matters to the general public".

A small sample size needs a more significant result if you want to draw a conclusion from it.*

• If your drug cures 30% of 10 people, the percentage of the general population cured could be anywhere between around 0% and 65% of people.

• If your drug cures 30% of 10000 people, you can be quite sure it actually cures around 30% of people (more specifically, between 29% and 31% of people).

• If your drug cures 100% of 10 people, you can be quite sure it would cure around at least 60% of people.

• If your drug cures 100% of 10000 people, you can be quite sure it actually cures around 100% of people.

When a 10000-person study says "we don't whether it cures anyone", we can be pretty sure that it cures between 0% and a very, very tiny percentage of the population. Whereas with a 10-person study with the same conclusion it could still cure a fairly large percentage. We just don't know yet.

*: this answer uses "significant" to mean "practically significant" not "statistically significant". As in "something that actually matters to the general public".