I would like to estimate the distribution of a very large population of known size but unknown mean and variance. I cannot assume anything about the shape of the underlying distribution (although I am relatively certain that it is not normal). However, I am certain that the values in the population are non-negative, non-zero integers. I believe that the distribution is naturally lumped in some way discretely rather than being continuously distributed over the entire range. I cannot sample the entire population but I would like to estimate the probability density function. I would also like some level of assurance of the correctness of the estimated distribution. What is the best way to go about this?
3
-
1$\begingroup$ Are you saying you cannot draw a sample that includes every element in the population, or that you cannot draw a sample such that every element in the population has equal probability of making it into the sample? $\endgroup$– fgreggCommented Jun 20, 2011 at 17:29
-
$\begingroup$ @fgregg: I CAN draw a sample such that every element in the population has equal probability of making it into the sample but it would prohibitively expensive computationally to draw a sample that includes every element in the population. $\endgroup$– SanjayCommented Jun 20, 2011 at 17:54
-
$\begingroup$ If you'll present us with a histogram of a reasonably large sample of the population, someone will probably have suggestions as to good candidate distributions against which you can test it, and how best to do so. $\endgroup$– rolando2Commented Jun 26, 2011 at 14:52
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Since the empirical distribution function of the sample should converge to the true population distribution function as the sample size approaches the population size, it seems to me that best estimate of the population pdf is the empirical pdf of the largest sample you an afford to draw.
Depending upon what the sample pdf looks like, you could consider summarizing the distribution with some functional form, particularly if you have some ideas about what generated the population.
