Looking for resources (papers, books) that explain the impact that non-random sampling has in test statistics The majority (if not all) of test statistics assume random sampling. Consequently, probability values obtained in t-tests, ANOVAs, regression, HLM, etc., are intrinsically linked to the assumption of random sampling.
However, in social sciences, it is often the case that random sampling is not possible, as you resort to convenience sampling (e.g., depressed individuals, autistic individuals, etc). Do you know of accessible - meaning not heavily mathematical - resources to better understand how results can be interpreted in light of convenience sampling and the overall trends/impact non-random sampling has in the interpretation of results?
 A: Missing data is an important source of non-random sampling, in particular if you are running multivariate analysis, where some particular regressor might be missing. This paper is a gentle introduction to the issue. This is a great book.
In the context of panel data, I would start by looking at the literature on unbalanced panels. This is the case of some units not having the full set of observations. This is directly related to non random samples, as the patter of selection might be non-random. Here is a nice presentation on the topic. Here is a fairly recent review paper.
A common solution to non-random samples is the use of weights. I find this introductory note, by the UK Data Service, very illuminating. It describes the issue and possible solutions.
The issue of clustering, related to the above, is also of major importance. Some datasets are constructed based on clustered sampling. This article, albeit a bit mathematical, deals with robust inference when non-random sampling is due to clusters.
A: However trivial it may seem, Wikipedia has a nice article on non-probability sampling. The bottom line of the article is that non-probability sampling techniques are not intended to be used to infer from the sample to the general population in statistical terms.
As far as statistical inference in case of complex probability sampling designs beyond simple random sampling is concerned Sharon L. Lohr's SAMPLING: DESIGN AND ANALYSIS seems to be a good resource.
