Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I would like to estimate the distribution of a very large population of known size but unknown mean and variance. I cannot assume anything about the shape of the underlying distribution (although I am relatively certain that it is not normal). However, I am certain that the values in the population are non-negative, non-zero integers. I believe that the distribution is naturally lumped in some way discretely rather than being continuously distributed over the entire range. I cannot sample the entire population but I would like to estimate the probability density function. I would also like some level of assurance of the correctness of the estimated distribution. What is the best way to go about this?

share|improve this question
1  
Are you saying you cannot draw a sample that includes every element in the population, or that you cannot draw a sample such that every element in the population has equal probability of making it into the sample? –  fgregg Jun 20 '11 at 17:29
    
@fgregg: I CAN draw a sample such that every element in the population has equal probability of making it into the sample but it would prohibitively expensive computationally to draw a sample that includes every element in the population. –  Sanjay Jun 20 '11 at 17:54
    
If you'll present us with a histogram of a reasonably large sample of the population, someone will probably have suggestions as to good candidate distributions against which you can test it, and how best to do so. –  rolando2 Jun 26 '11 at 14:52
add comment

1 Answer

Since the empirical distribution function of the sample should converge to the true population distribution function as the sample size approaches the population size, it seems to me that best estimate of the population pdf is the empirical pdf of the largest sample you an afford to draw.

Depending upon what the sample pdf looks like, you could consider summarizing the distribution with some functional form, particularly if you have some ideas about what generated the population.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.