Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

In my experiment I compute the average latency of operations per second. I would like to define N, i.e: how many times do I need to run my experiment to compute a close-to-real average latency?

I figured that I could apply the CLT here, because If I repeat the same experiment 1000 times and plot a histogram, I get a normal distribution curve. Is the central theorem useful in my case? In the definitions I found, to be able to compute an estimated mean with a certain error, one needs to know the variance beforehand, and I don't know it.

share|improve this question
3  
This is more ore less textbook stuff. The key word here is t-statistic. Central limit theorem is certainly useful, if your latency has finite variance. If it does not, then it becomes trickier. –  mpiktas Nov 22 '12 at 13:28
1  
You do need to know how close is close enough in regards to "real average latency" to even approach this question. Usually what we do in these cases is estimate the population variance using the sample estimate of the population variance (usually just called sample variance). –  rpierce Jul 24 '13 at 0:00
add comment

2 Answers

Perhaps you can bound your variance. Suppose, for example, that you know your data must be in the range $[a,b]$. Then Popoviciu's inequality bounds your variance by $\sigma^2 \le (1/4)(b-a)^2$. Using the upper bound in the formulas you found will be a bit of overkill, but it should satisfy your requirements.

share|improve this answer
add comment

CTL is all about independent and identically distributed (i.i.d.) random variables, with finite mean and variance. I edit the answer just to add that you don't have to know your parameter , but be sure that this parameter is finite and identical along your runs.

In order to estimate the parameter mean with unknownk variance you can build an interval using the t-student as

$$ \bar X - t_{n-1,1-\alpha} \frac{S}{\sqrt{n}} $$

where $t$ is the value of a t-student with $n-1$ degrees of freedom with $1-a$ confidence level.

share|improve this answer
    
Do you have any pointer on how to calculate the number of experiment to have 95 accuracy without knowing the variance? thanks –  Djellel Eddine Nov 22 '12 at 13:16
2  
-1, Answers should be answers, this is more like an unfinished comment. Please elaborate more. –  mpiktas Nov 22 '12 at 13:31
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.