60

There is no minimum sample size for the t test to be valid other than it be large enough to calculate the test statistic. Validity requires that the assumptions for the test statistic hold approximately. Those assumptions are in the one sample case that the data are iid normal (or approximately normal) with mean 0 under the null hypothesis and a variance ...


54

I would like to list another intuitive example. Suppose I tell you I can predict the outcome of any coin flip. You do not believe and want to test my ability. You tested 5 times, and I got all of them right. Do you believe I have the special ability? Maybe not. Because I can get all of them right by chance. (Specifically, suppose the coin is a fair coin, ...


51

I agree with many of the other answers here but think the statement is even worse than they make it out to be. The statement is an explicit version of an implicit claim in many shoddy analyses of small datasets. These hint that because they have found a significant result in a small sample, their claimed result must be real and important because it is '...


42

If the population is known to be normal, a 95% confidence interval based on a single observation $x$ is given by $$x \pm 9.68 \left| x \right| $$ This is discussed in the article "An Effective Confidence Interval for the Mean With Samples of Size One and Two," by Wall, Boen, and Tweedie, The American Statistician, May 2001, Vol. 55, No.2. (pdf)


38

With all deference to him, he doesn't know what he's talking about. The t-test was designed for working with small samples. There isn't really a minimum (maybe you could say a minimum of 3 for a one-sample t-test, IDK), but you do have a concern regarding adequate power with small samples. You may be interested in reading about the ideas behind compromise ...


37

Wayne has addressed the "30" issue well enough (my own rule of thumb: mention of the number 30 in relation to statistics is likely to be wrong). Why numbers in the vicinity of 1000 are often used Numbers of around 1000-2000 are often used in surveys, even in the case of a simple proportion ("Are you in favor of $<$whatever$>$?"). This is done so ...


36

[I'll assume from the discussion in your question that you're happy to accept as fact that if $Z_i, i=1,2,\ldots,k$ are independent identically distributed $N(0,1)$ random variables then $\sum_{i=1}^{k}Z_i^2\sim \chi^2_k$.] Formally, the result you need follows from Cochran's theorem. (Though it can be shown in other ways) Less formally, consider that if ...


35

"It's counterintuitive, but the smaller the sample size of a clinical test, the more significant its results are. The differences in a sample of 20 people may be more significant than in a sample of 10,000 people. If we need such a sample, there is a risk of being wrong. With 10,000 people, when the differences are small, sometimes they don't exist." I ...


33

As mentioned in existing answers, the main issue with a small sample size is low statistical power. There are various rules of thumb regarding what is acceptable statistical power. Some people say 80% statistical power is reasonable, but ultimately, more is better. There is also generally a trade-off between the cost of getting more participants and the ...


29

There are several issues here. Typically, we want to determine a minimum sample size so as to achieve a minimally acceptable level of statistical power. The sample size required is a function of several factors, primarily the magnitude of the effect you want to be able to differentiate from 0 (or whatever null you are using, but 0 is most common), and ...


29

Think about it in terms of proportions. Let's say that preferring an orange is a success, while preferring an apple is a failure. So your mean success rate is $\mu = \frac{\text{# of sucesses}}{n}$ or in this case .6 The standard error of this quantity is estimated to be $\sqrt{\frac{\mu(1-\mu)}{n}}$. For a small sample size (i.e. 10), the standard error is ...


28

Sure there is. Use a Bayesian paradigm. Chances are you have at least some idea of what $\mu$ could possibly be - for instance, that it physically cannot be negative, or that it obviously cannot be larger than 100 (maybe you are measuring the height of your local high school football team members in feet). Put a prior on that, update it with your lone ...


25

The short answer is that your conjecture is true when and only when there is a positive intra-class correlation in the data. Empirically speaking, most clustered datasets most of the time show a positive intra-class correlation, which means that in practice your conjecture is usually true. But if the intra-class correlation is 0, then the two cases you ...


24

Short answer: Yes, you can use ID as random effect with 6 levels. Slightly longer answer: The @BenBolker's GLMM FAQ says (among other things) the following under the headline "Should I treat factor xxx as fixed or random?": One point of particular relevance to 'modern' mixed model estimation (rather than 'classical' method-of-moments estimation) is ...


24

(I think the phrasing is deliberately provocative.) If you have 10 observations and want to show that their mean is not zero, it will have to be quite a bit different from 0 if you want to have any kind of chance (power) of detecting the difference. If you have a trillion observations and want to show that their mean is not 0, the mean could be just a tiny ...


23

For factor analysis (not principal component analysis), there is quite a literature calling into question some of the old rules of thumb on the number of observations. Traditional recommendations – at least within psychometrics – would be to have at least $x$ observations per variable (with $x$ typically anywhere from $5$ to $20$) so in any case $n \gg p$. ...


22

Note gung's question; it matters. I will assume that the treatment was the same for every tank in the treatment group. If you can argue the variance would be equal for the two groups (which you would typically assume for a two sample t-test anyway), you can do a test. You just can't check that assumption, no matter how badly violated it might be. The ...


21

You can actually measure whether your sample size is "large enough". One symptom of small sample size being too small is instability. Bootstrap or cross validate your PCA: these techniques disturb your data set by deleting/exchanging a small fraction of your sample and then build "surrogate models" for each of the disturbed data sets. If the surrogate ...


21

Statistical tests do not make assumptions about sample size. There are, of course, differing assumptions with various tests (e.g., normality), but the equality of sample sizes is not one of them. Unless the test used is inappropriate in some other way (I can't think of an issue right now), the type I error rate will not be affected by drastically unequal ...


21

Consider tossing a coin which you suspect may come up heads too often. You perform an experiment, followed by a one tailed hypothesis test. In ten tosses you get 7 heads. Something at least as far from 50% could easily happen with a fair coin. Nothing unusual there. If instead, you got 700 heads in 1000 tosses, a result at least as far from fair as that ...


21

You should rarely trust any single significant result. You didn't say why you were using a one-tailed instead of a two-tailed test, so hopefully you have a good reason for doing so other than struggling to be able to claim a statistically significant outcome! Setting that aside, consider the following from p. 261 of Sauro, J., & Lewis, J. R. (2016). ...


19

Some books state a sample size of size 30 or higher is necessary for the central limit theorem to give a good approximation for $\bar{X}$. This common rule of thumb is pretty much completely useless. There are non-normal distributions for which n=2 will do okay and non-normal distributions for which much larger $n$ is insufficient - so without an explicit ...


19

The standard deviation is as applicable here as anywhere else: it gives useful information about the dispersion of the data. In particular, the sd divided by the square root of the sample size is one standard error: it estimates the dispersion of the sampling distribution of the mean. Let's calculate: $$3.2\% / \sqrt{10000} = 0.032\% = 0.00032.$$ That's ...


18

I don't prefer to think of this as a power issue, but rather ask the question "how large should $n$ be so that the apparent $R^2$ can be trusted"? One way to approach that is to consider the ratio or difference between $R^2$ and $R_{adj}^{2}$, the latter being the adjusted $R^2$ given by $1 - (1 - R^{2})\frac{n-1}{n-p-1}$ and forming a more unbiased ...


18

The question you are asking is different from "convergence diagnostics". Lets say you have run all convergence diagnostics(choose your favorite(s)), and now are ready to start sampling from the posterior. There are two options in terms of effective sample size(ESS), you can choose a univariate ESS or a multivariate ESS. A univariate ESS will provide an ...


17

I typically use a 15:1 rule (ratio of min(events, non-events) to number of candidate parameters in the model). More recent work found that for a more rigorous validation 20:1 is needed. More information may be found in my course handouts linked from http://biostat.mc.vanderbilt.edu/rms, in particular an argument for a minimum sample size of 96 just to ...


17

Yes, the Mann-Whitney test works fine with unequal sample sizes.


17

Note that at each observation position ($i=1, 2, ..., n$) we can choose any of the $n$ observations, so there are $n^n$ possible resamples (keeping the order in which they are drawn) of which $n!$ are the "same sample" (i.e. contain all $n$ original observations with no repeats; this accounts for all the ways of ordering the sample we started with). For ...


17

This concept is a consequence of the law of large numbers. From Wikipedia, According to the law, the average of the results obtained from a large number of trials should be close to the expected value, and will tend to become closer as more trials are performed. Results from a small sample may be farther from the expected value than that from a larger ...


16

In an attempt to figure out the minimum number of groups for a multilevel model I looked at the book Data Analysis Using Regression and Mulitilevel/Hierarchical models by Gelman and Hill (2007). They appear to address this topic in Chapter 11, Section 5 (page 247) where they write that when there are < 5 groups then multilevel models typically add little ...


Only top voted, non community-wiki answers of a minimum length are eligible