2
$\begingroup$

I am looking for the sample size of a relatively abstract problem.

Here is an example to make it more tangible. The plan is to investigate a new type of motorcycle which can react to roadway damages (by e.g. avoiding them automatically) and therefore test the highways of a specific country for damages. At the moment I dont exactly know which road damages the motorcycle and its system reacts to and how many damages there exactly are, but based on experience damages should appear every 5 miles. The mentioned type of motorcycle is not available yet, but I would like to know how many miles I would have to drive to ensure to cover the most critical damages (statistical certainty e.g. 95%).

What would be a appropriate way to calculate the sample size with those uncertainties?

Improve this question
$\endgroup$
6
  • $\begingroup$ What will be the outcome of your investigation? The percentage of road damages that the motorcycle avoids? $\endgroup$
    – elmo
    Nov 16, 2016 at 11:49
  • $\begingroup$ How well does the avoiding system work? What needs to be improved? $\endgroup$
    – Kamil
    Nov 16, 2016 at 12:04
  • $\begingroup$ In order to determine the sample size, "how well does it work" needs to be quantified. One example is as I suggested "How much percent of the road damages does it avoid?", but there are obviously also other options. After you have chosen how you want to measure "how well does it work", I might be able to further assist you in the sample size determination. $\endgroup$
    – elmo
    Nov 16, 2016 at 12:48
  • $\begingroup$ We can use the percentage of avoided road damages as an outcome. Another (general) possible result could be "how many road damages are there" and therefore "how big has my sample need to be" $\endgroup$
    – Kamil
    Nov 16, 2016 at 13:15
  • $\begingroup$ The sample size determination will have to be based on a number of road damages anyway (e.g. "You need to drive past 200 road damages in order to ..."), therefore the amount of miles you need to drive can only be estimated using "required road damages"x5. As a secondary outcome in the study you could measure the distance between road damages, e.g. "3miles between start and first damage", "5miles between first damage and second damage" etc and analyse this seperately. I will add an answer regarding the computation of the sample size in the next hours. $\endgroup$
    – elmo
    Nov 16, 2016 at 13:25

1 Answer 1

Reset to default
1
$\begingroup$

Statistically speaking, you want to calculate the required sample size in order to estimate a single proportion ("What's the percentage of road damages that the motorcycle will avoid") with a certain precision ("How exact should the estimation be"), given a certain significance level ("error probability").

The ingredients you need for to calculate the sample size are therefore:

  1. Significance level $\alpha$. Usually this is set to $0.05$, but nothing is keeping you from choosing a different value.
  2. An initial estimate for the proportion, let's call it $p$. If you happen to have some data from a pilot study or the test ground or just a strong intuition, you can use it. If not, you can set $p=0.5$, because this will results in the biggest sample size and is therefore your "worst-case-scenario".
  3. The precision $\delta$. This will determine how precise your estimate of the proportion will be, i.e. how long the width of the confidence interval will be.

The formula to calculate the required sample size is given by

$n \geq (\frac{Q^{N(0,1)}(1-\frac{\alpha}{2})}{\delta})^2*p*(1-p)$,

where $Q^{N(0,1)}(1-\frac{\alpha}{2})$ denotes the $1-\frac{\alpha}{2}$ quantile of the standard normal distribution.

I will present one scenario here and aid you with the interpretation.

Assume you set $\alpha=0.05$, meaning you allow for $5$% error probability, $p=0.5$, since you have no idea about how the motorcycle will perform and $\delta = 0.015$, meaning you want to know the percentage of road damages that it avoids within $3$%.

Then the above formula yields $n\geq 4269$.

This means: If you drive the motorcycle around until you encountered $4269$ road damages and record for each road damage, whether or not the motorcycle avoided it, you will be able to estimate the total (=theoretical) proportion that the motorcycle avoides with a precision of at least $3$% while having an "error probability" of $5$%.

I deliberately wrote "a precision of at least $3$%", because the further away the total proportion is from $0.5$, the more precise your estimation will be.

The results that you will get after your study will be a confidence interval, which might look like this: $[0.44; 0.47]$. This would be interpreted (roughly) as "There is a $95$% chance that the true proportion of road damages that the motorcycle avoids is between $44$% and $47$%".

Improve this answer
$\endgroup$
14
  • $\begingroup$ Thanks a lot! Do you take into account if I investigate the road damages in USA (total amount of highway miles 50,000) or UK (2,500 total highway miles)? $\endgroup$
    – Kamil
    Nov 16, 2016 at 15:40
  • $\begingroup$ No I did not - the sample size can say nothing about the actual miles you need to drive. At most you can multiply the number by e.g. 5 to get an estimate of how many miles it will amount to. Ultimately it's about the amount of road damages that you encountered. Whether you encounter them in the US in 10,000 miles or in the UK in 15,000 miles or in Central Africa in 1,000 miles does not matter. $\endgroup$
    – elmo
    Nov 16, 2016 at 15:50
  • $\begingroup$ If I was able to answer your question, I kindly ask for an upvote and my answer to be accepted. If you have more questions, feel free to ask. $\endgroup$
    – elmo
    Nov 16, 2016 at 15:53
  • $\begingroup$ I dont totally understand the country issue. At the end of the day I want to investigate how my motorcycle works in the US, UK or Central Africa. - So if USA has 50,000 total highway miles and UK just 2,500 miles, I feel like my sample size in the US would have to be way bigger then UK. (after 2,500 miles in the UK Ive seen every road damage, therefore I'd perfectly know how my motorcycle works at every corner... but after 2,500 miles in the US I probably have seen just a very small portion $\endgroup$
    – Kamil
    Nov 18, 2016 at 9:47
  • $\begingroup$ - road damages directly depend on the investigated country (road maintenance, road technology, and so on). Therefore i need to take into account where I'm investigating - last thing: If I want to know the percentage of road damage in the specific country (and not primarily how well the system works) I feel like the total highway network (US 50000 - UK 2500 miles) does matter $\endgroup$
    – Kamil
    Nov 18, 2016 at 9:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.