How do I find a confidence interval for rate defective from my sample taken without replacement? I have a population of let's say 10000 discrete data points. I want to assess this population on a certain parameter (let's say error rate) based on a sample. I don't want to check the error rate on the entire population. I want to calculate this error rate on a sample only.
How do I decide if a 10% sample is enough or 20% sample is enough or 30% sample is enough?
 A: What you are asking for is a "power analysis" and the answer depends on (among other things) how precisely you want to estimate the error rate.
Here's what you need to do:
Step 1: Decide what level of confidence you are working with (95%, 99% etc) and find the two tailed z value associated with that level (e.g. 1.96 for 95% confidence). This is "z."
Step 2: Make a guess as to what the error rate is likely to be (if you have no idea, just guess 50% since that's the most conservative). Call that "p."
Step 3: Estimate the standard deviation of p. If p is a proportion (e.g. an error rate) this will just be $\sqrt{p(1-p)}$ and call that value $\sigma$. If you guessed 50% in step 3 this will just be 50%, which is as big as it can ever be.
Step 4: Decide how "wide" (in percentages) you are willing to let the confidence around your estimate be. In other words, would you be happy if your estimate had confidence intervals that were plus or minus 5%? Or would you only be happy if they were plus or minus 2%. Call this value E.
Plug these numbers into the formula for power analysis for a proportion
$$n=(\frac{z*\sigma}{E})^2 $$.
"n" is the number of data points you need to sample to estimate the error rate with confidence intervals that are "E"% wide at your chosen level of confidence.
For example, if you were working at 95% confidence and were willing to let the CIs be plus or minus 5% then the answer would be
$$n=(\frac{1.96*.5}{.05})^2=384.16 $$.
Since you can't sample fractional units you would have to sample at least 385 data points to get an estimate this accurate.
Note that it's the NUMBER of data points you sample, not their proportion relative to the populating that matters. You would need the same size sample to generalize about a population of 1,000 or a population of one million. This is a consequence of the central limit theorem.
A: The proportion (10%, 20%, 30%, etc.) you need to take
depends on how closely you need to approximate the
defective rate.
For example, suppose the population size is 10,000 with
$d = 0.13$ defective (that is, 1300 defective and 8700 good). Then you sample 10% (1000). obtaining 1000 estimates $\hat d_{.10}$ of the defective rate. Because
of the large sample sizes the estimates will be nearly
normally distributed, so we can get an idea of the
length of the CI $(0.1294, 0.1306)$ with the usual normal formula.
The exact computations can be found using the
mean and variance of the appropriate hypergeometric
distribution. I used a simulation below:
set.seed(1234)
m = 10^6;  N = 10^4; d = .13
d.10 = rhyper(m, N*d, N*(1-d), N*.1)/(N*.1)
mean(d.10)+ qnorm(c(.025,.975))*sd(d.10)/sqrt(N*.1)
[1] 0.1293771 0.1306276

For a 30% sample (of size 3000), we get a slightly shorter CI $(0.1299, 0.1301).$
d.30 = rhyper(m, N*d, N*(1-d), N*.3)/(N*.3)
mean(d.30)+ qnorm(c(.025,.975))*sd(d.30)/sqrt(N*.3)
[1] 0.1298128 0.1301805

You should probably explore several larger and smaller defective rates.
