Validating new estimators Can anyone please tell me whether there it be a problem if I validate an estimator by taking samples from one population. This is how most of the estimators for respondent driven sampling has been validated. Is there any drawback in this method?
I personally feel that by using just one population there will be a problem.
 A: Statistical results are only every applicable to the population they have been tested on. If an estimator is validated on a single population then you can only assert your level of confidence in its ability to provide insight for that population. 
In any new population you have no certainty that all of the properties observed in your validated population apply. Some examples:


*

*There may be some of the properties lie outside the range observed in the valuation set, so you have no guarantee your model continues to hold under extrapolation. The smaller the extrapolation the less of a problem this is likely to be. 

*there may be other factors not captured in the original consideration that influence the applicability of the model. Since these were not captured there is zero understanding of how these may differ in any new population

*There may be differences in how multiple factors interact and correlate with each other. This in turn will change any model of how an estimator interacts with the value you are estimating.

*Bias is a big problem in any form of sampling and certainly with respondent driven sampling. Respondent driven sampling is inherently biased towards sampling people who respond and exclude people who don't. Online product reviews (for simplicity we consider only honest ones here) tend to come mostly from people very happy or very upset about the product (and so motivated to respond) and so is not good at discriminating between middle of the road products. In health surveys the sickest people often have the least ability to respond. The demographics will also depend strongly on the methods used to recruit and raise awareness. Incentives will further distort the demographics. Any validation of estimators derived on respondent driven data should be backed up with a robust population characteristic comparison as a minimum. 
** 2nd Update**
some references (including reports on attempts to ameliorate some of teh issues)
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3277908/
https://openarchive.ki.se/xmlui/handle/10616/41378
https://www.sciencedirect.com/science/article/pii/S0376871614009375
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.715.9315&rep=rep1&type=pdf
http://www.pnas.org/content/107/15/6743
https://pdfs.semanticscholar.org/0262/e9e917a422d381a49cbd71d727361a5095da.pdf
A common issue for all types of sampling is the problem of understanding how the population used to model, or for previous validations, compares to a new population you want to test on. With respondent driven sampling then the new population could be radically different if you try a different outreach method. It is important to identify what demographic information is critical to your query, then compare this between existing studied populations and any new proposed populations. This is never going to give a definitive answer, but it is all about gauging the level of confidence you can place in your analysis and inference. The more you know, the better you can estimate the risk you are taking.
