In the context of hypothesis tests, poor finite sample properties usually mean that the actual rejection rate of the test differs from the nominal one.
Recall that the nominal one is the level at which you are testing, often 5%. That means that you expect to reject 5% of all correct null hypotheses. If the actual rejection frequency differs from the nominal one, we have what is known as a size distortion.
Often, this distortion arises because we compute critical values for our test based on asymptotic considerations (often, the central limit theorem), i.e., an asymptotic distribution of the test statistic under the null (typically because, except in special cases, the finite sample distribution of the test statistic is not known or not tractable). That is, the distribution that the test statistic would follow if the null is true and if we have infinitely many observations. Clearly, in practice, we don't, so the statistic will follow its (unknown) finite sample distribution.
[So the answer to your question is basically yes, many dummies will lead to a low effective sample size (a.k.a. eat up degrees of freedom) - there no such thing as a unique threshold for "small" sample sizes. E.g., in problems with many parameters, such as long regressions, more data is needed for good inference than in simpler problems.]
Hence, its quantiles will differ from the quantiles of the asymptotic distribution, where the latter are used to get critical values. Thus, the test statistic will not exceed the critical value (i.e., reject) as often as expected from the nominal level.
Here is a simple example (a little Monte Carlo study that simulates the actual rejection rate of a test):
mean(replicate(10000, abs(t.test(rchisq(10, df=1)-1)$statistic) > qnorm(.975)))
What does the code do? It
- simulates 10 r.v.s from a (very skewed and) demeaned $\chi^2_1$-distribution (recall that the mean of a $\chi^2_k$ distribution is $k$, so that subtracting 1 here produces a mean zero distribution)
- finds the usual t-statistic for the (correct) null that the expected value of the distribution from which we sample is zero (that is what
t.test does without specifying further options)
- checks if the (absolute value of the) test statistic is larger than the 5%-critical value of a two-sided t-test (
qnorm(.975)) (hence, if we reject at nominal level 5%) based on the normal distribution that the t-statistic follows asymptotically (see e.g. Is there a version of the central limit theorem which lets me replace Var(X) with the sample variance?),
- it does that 10,000 times, producing 1s for rejections and 0s for acceptances
- and takes an average to produce a finite-sample rejection rate.
Depending on the seed, this code will return values of around 0.17, where, if the test had good finite-sample properties, we would see 0.05. In other words, instead of rejecting every 20th correct null hypothesis, as one might think it does based on the nominal level, it rejects every 6th, thus leading us to reject the null (quite far) too often.
[In practice, information such as the one used here that we sample from a certain distribution is not available, so that we will not easily be able to work with more accurate critical values. If I had run the code with, say, draws from a symmetric mean-zero such as a t-distribution with small degrees of freedom, we would have still seen some size distortion, but much less.]
[Is a size distortion that leads to rejecting too infrequently bad, too - after all, it leads to a smaller type I-error rate? I'd say yes, because rejection rates do not change discontinuously at the margin of null and alternative - hence, if you reject too infrequently if the null is true, you'll also reject less frequently when it is false, i.e., you'll have low power.]
If you replace 10 in the code with, say, 1000, you will get values much closer to the nominal level 0.05, illustrating that the problem is a finite-sample one.
Trying this for different $n$ would produce something like this:
n <- c(10,20,50,100,200,500,1000,10000)
mcn <- lapply(n, function(i) mean(replicate(10000, abs(t.test(rchisq(i, df=1)-1)$statistic) > qnorm(.975))))
plot(n, unlist(mcn), type="o", col="salmon", lwd=2, ylim=c(0,0.2), ylab="rejection rate")
abline(h=0.05, lty=2)
