# Assessing the representativeness of population sampling

I am looking for some suggestions about assessing the representativeness of a particular dataset I am analyzing.

In this dataset I am looking at the relationship between two variables (e.g., X and Y) in a population that is split into five distinct blocks. The main problem is that the data is based upon reports from the public, so some blocks have much more data than others.

The goal is to assess whether the relationship between X and Y differs between the blocks, but also to determine how reliable such estimates are given that we do not have a truly random sample of the overall population.

Any suggestions appreciated.

Thanks

• @user3136 What kind of variables are you considering, and what kind of relationship are you interested in? – chl Mar 2 '11 at 11:42
• The two variables are both continuous, a simple correlation or perhaps a regression per block would be enough to summarize the relationship. I have calculated these already, but my problem is that the sampling is not strictly random so I am not convinced how much I can trust any p-values, confidence intervals etc. – user3136 Mar 2 '11 at 12:10
• If the sample isn't random, it's going to be difficult to make any firm conclusions unless you create a model for the non-randomness (e.g., certain demographic groups are underrepresented). – Charlie Mar 2 '11 at 15:30
• Following on from Charlie, you might be in the happy situation where the non-random selection process is the same for each block. Then your original question 'is the X Y relationship different between blocks?' is actually addressable by the usual methods despite the fact that each method will be arbitrarily wrong about what that relationship actually is... – conjugateprior Mar 2 '11 at 18:58
• I like Charlie's and Conjugate Prior's remarks. More prosaically, the fraction of the population that was sampled within each block might have some bearing on the reliability of estimates. – rolando2 Mar 3 '11 at 17:59