Sometimes in reports I include a disclaimer about the p-values and other inferential statistics I've provided. I say that since the sample wasn't random, then such statistics would not strictly apply. My specific wording is usually given in a footnote:
"While, strictly speaking, inferential statistics are only applicable in the context of random sampling, we follow convention in reporting significance levels and/or confidence intervals as convenient yardsticks even for nonrandom samples. See Michael Oakes's Statistical inference: A commentary for the social and behavioural sciences (NY: Wiley, 1986).
On a couple of occasions--once for a peer-reviewed paper, once or twice in a non-academic setting--the editor or reviewer objected to this disclaimer, calling it confusing, and felt that the inferential findings should simply stand as written (and be given the mantle of authority). Has anyone else encountered this problem and found a good solution? On the one hand, people's understanding of p-values is generally dismal, even in the context of random sampling, so perhaps it doesn't matter much what we say. On the other, to contribute further to misunderstandings seems to make one part of the problem. I should add that I frequently deal with survey studies, where random assignment does not apply and where Monte Carlo simulations would often fail to address the issue of representativeness.