Sample Size with Uncertainties I am looking for the sample size of a relatively abstract problem.
Here is an example to make it more tangible. The plan is to investigate a new type of motorcycle which can react to roadway damages (by e.g. avoiding them automatically) and therefore test the highways of a specific country for damages. At the moment I dont exactly know which road damages the motorcycle and its system reacts to and how many damages there exactly are, but based on experience damages should appear every 5 miles. The mentioned type of motorcycle is not available yet, but I would like to know how many miles I would have to drive to ensure to cover the most critical damages (statistical certainty e.g. 95%).
What would be a appropriate way to calculate the sample size with those uncertainties?
 A: Statistically speaking, you want to calculate the required sample size in order to estimate a single proportion ("What's the percentage of road damages that the motorcycle will avoid") with a certain precision ("How exact should the estimation be"), given a certain significance level ("error probability").
The ingredients you need for to calculate the sample size are therefore:


*

*Significance level $\alpha$. Usually this is set to $0.05$, but nothing is keeping you from choosing a different value. 

*An initial estimate for the proportion, let's call it $p$. If you happen to have some data from a pilot study or the test ground or just a strong intuition, you can use it. If not, you can set $p=0.5$, because this will results in the biggest sample size and is therefore your "worst-case-scenario".

*The precision $\delta$. This will determine how precise your estimate of the proportion will be, i.e. how long the width of the confidence interval will be.  


The formula to calculate the required sample size is given by
$n \geq (\frac{Q^{N(0,1)}(1-\frac{\alpha}{2})}{\delta})^2*p*(1-p)$,
where $Q^{N(0,1)}(1-\frac{\alpha}{2})$ denotes the $1-\frac{\alpha}{2}$ quantile of the standard normal distribution. 

I will present one scenario here and aid you with the interpretation. 
Assume you set $\alpha=0.05$, meaning you allow for $5$% error probability, $p=0.5$, since you have no idea about how the motorcycle will perform and $\delta = 0.015$, meaning you want to know the percentage of road damages that it avoids within $3$%. 
Then the above formula yields $n\geq 4269$. 
This means: If you drive the motorcycle around until you encountered $4269$ road damages and record for each road damage, whether or not the motorcycle avoided it, you will be able to estimate the total (=theoretical) proportion that the motorcycle avoides with a precision of at least $3$% while having an "error probability" of $5$%.  
I deliberately wrote "a precision of at least $3$%", because the further away the total proportion is from $0.5$, the more precise your estimation will be. 
The results that you will get after your study will be a confidence interval, which might look like this: $[0.44; 0.47]$. This would be interpreted (roughly) as "There is a $95$% chance that the true proportion of road damages that the motorcycle avoids is between $44$% and $47$%". 
