Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I've implemented Shlosser's estimator for the number of distinct values in a population given a sample. Now I want to make sure my algorithm is correctly implemented. Are there any example sets out there that would allow me to verify my computation matches the canonical value?

share|improve this question
add comment

1 Answer 1

up vote 2 down vote accepted
+50

I may be mistaken, but it seems that Schlosser estimator is a pretty obscure technique: you might get more answers by explaining the very popular version acceptable for non-skewed data -- a jacknife estimator.

Since I was trained in Biostatistics, all the examples that come to mind would be low-incidence diseases (e.g., rare genetic diseases) with "inconsistent" predictors (genetic, phenotypic, etc.)

For instance, I used to help with a non-profit that supports CFC Syndrome; they have collected almost every incidence of this rare genetic disease in the world to support their lobbying and research fund-raising efforts. I do not remember if their dataset is publicly available, but this would be an instance of a highly irregular distribution of incidence in the general pop, where many of the phenotypic data they have collected would not directly predict the disease.

Another example might be datasets from genome-wide association ("GWAS") studies on certain low-frequency diseases (e.g., Goldstein group at Duke). The big problem with this example is that associations between genes and associated disease do not really have a "ground truth" except with very small samples of the total affected population (say, only small number of AIDS-immune patients have been "found" in W. Europe, yet we have attributed the disease to a specific set of genes that seem to be associated with the immunity).

Conclusion: you might be making up your own "fixture" dataset rather than trying to find acceptable data from science or business ... :/

share|improve this answer
    
Thanks for the reply! My data is pretty skewed, which is why I'm considering the Shlosser estimator. Though I may end up not using Shlosser, nevertheless I want to make sure my Shlosser implementation is correct, which is why I'm looking for example sets. –  Jon Smark Nov 30 '12 at 16:21
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.