Resampling small datasets - Issue of overcounting? Hypothetical situation here, as techniques like bootstrapping often fail for very small datasets.
Taking bootstrapping as an example nonetheless. we can easily compute the number of possible bootstrap (re)samples.
A nice answer is given by @whuber here:
Amount of Possible Bootstrap Samples
Suppose for a moment that bootstrapping is perfectly valid for low sample sizes. Now, say we have $n = 5$. From the posted solution above, it is found that there are a total of 126 possible boostrap resamples that can be drawn.
In bootstrapping, we usually take a large number of replications (10000 for instance). Taking this many replicates for such a small dataset, like the one above, seems bizarre, since resamples will be counted multiple times.
Question: Does this really matter? If yes. what is the effect on inference?
 A: When bootstrapping, we are assuming that the sample is representative of the population.
The whole purpose of bootstrapping is to estimate a sampling distribution and infer the likely standard errors and confidence intervals for the population as a whole.
However, the issue with small sample sizes is that bias is more likely to exist relative to large samples - and (contrary to popular belief) bootstrapping does not fix this bias or remedy the issue of small sample sizes.
For instance, suppose one were to roll a dice five times. The numbers 4, 5, 6, 6, 6 are obtained, for an average of 5.4.
If one were to roll a dice a hundred times, an average closer to 3.5 could be expected - which is the theoretical mean.
However, small samples have a higher chance of deviating significantly from the population mean and so bootstrap sampling will not remedy this by virtue of simply generating more observations.
A: The issue with small sample sizes is not that you will repeat bootstrap samples but that the original small sample might not be so representative of the population.
Let’s obtain a sample of coin flips from a fair coin, so the true population is $Binom(1, 0.5)$, and let’s use your small sample size of $n=5$. In R…
set.seed(314) # For pi
x <- rbinom(5, 1, 0.5)

I get four $0$s (heads) and one $1$ (tails), which means a $20\%$ chance of tails, rather than the correct $50\%$. When we go to bootstrap this sample, we are telling the bootstrap procedure to sample from a $Binom(1, 0.2)$ distribution, which is quite a bit different from the true $Binom(1, 0.5)$ population.
When the sample size is larger, we are less likely to have a sample that is so drastically different from the population.
A: Many bootstrap replications are drawn in order to approximate (in a standard situation) the distribution of i.i.d. samples from the empirical distribution. Now if you can get this distribution explicitly, which is possible with a small sample as you can list all possible samples and their probabilities, it isn't necessary to approximate it by a much larger number of random bootstrap samples. Instead you can just use the full distribution of bootstrap samples (obviously potential problems with a lack of representativity of your small sample as mentioned in other answers still exist, but I believe this was not the question).
Note by the way that i.i.d. sampling from the empirical distribution will produce uniform probabilities over ordered rather than distinct samples, meaning that if you want to emulate the true bootstrap distribution by a set of samples, you will need some repetition of most of the 126 distinct samples. This would be approximated by randomly taking a very large number of bootstrap samples, so multiple counting of samples is not the issue here, rather what happens is you're using more computing power than necessary to do something less precise than possible with less computing effort.
