I have a giant array of integer numbers like this:
giantArray = [2048,128,512,512,128,8,....]
The content of this array changes every day and I need to estimate its mean really quick. I have limited processing resource for this job. Moreover, there is no guarantee that distribution and standard deviation of the array remain the same every day. The smallest integer in the array is 8 and the largest one is 2048. All other integers are the power of two numbers.
The size of the array is about 500 million entry and I cannot afford to read it all. Therefore, I think, I need to use simple random samples (SRS) to estimate the Sample mean.
However, Since I don't know my standard deviation I cannot calculate how big should be my minimum sample size for a given confidence level of 95% and error margin of +/- 8.
Do you know how can I calculate the standard deviation and then determined the minimum sample size?
Can I use this online algorithm to calculate standard deviation during sampling time and then use it to find out If I have sampled enough?