Hypothesis testing on exhaustive data If I'm not mistaken, statistical hypothesis testing does not hold when we have exhaustive data (ie the number of observations $N$ is equal to the size of the total population), as in that case we can directly assert if the null hypothesis is violated (for instance by computing the means of two subpopulations and checking whether or not they are equal). There is only a qualitative judgement to make, eg are means of 49.9 and 50.1 really different, but this judgement must also be made when using statistical tests.
I have three questions :


*

*Can one justify the use of statistical hypothesis testing by supposing that the observations are error-ridden, and making a (normality or iid) hypothesis on the form of the measurement error?

*Can one justify the use of statistical hypothesis testing by supposing that the observed population is a random occurence of some underlying law of interest? (this seems rather artificial)

*Would it be wrong to say that variable $y$ can be fairly well predicted by a set of variables $(x_1, x_2, x_3)$ if one proceeded by cross-validation on this exhaustive data set (eg if the cross-validated prediction error of a regression model is low)? Of course this is not a proper statistical test, but it seems like something one could say about the data.
Any help or reference greatly appreciated!
 A: The statistical theory is really about sampling from a process, not necessarily a population.  
Taking samples (simple random, or other) from a population is one type of process and is simple to think about and is therefore a common model taught in introductory statistics classes, but it is not the only process.
Another common process that we talk about is flipping a coin, or drawing ball from urns.  If I flip a coin 10 times that is not really a simple random sample of all the times that I will ever flip that coin (though it could be the entire population for that coin if I spend it without ever flipping it again), but we still use that as an example because while it is not an SRS from a population, it is reasonable to assume that the flips are iid from the same process (binomial with p=0.5 unless I am cheating somehow).
So to your specific questions:


*

*Assuming measurement error on a population is a random process and therefore you can use statistical inference techniques (the assumptions about distribution, iid, etc. will be crucial)

*This could go a couple of ways (but the short answer is yes).  I have seen descriptions of considering a population to be a random draw from a set of all the ways the population could have turned out (a hyper-population) and therefore justifies the inference (this approach is somewhat artificial as you state).  The other option is to focus on the process.  Consider a business where there were 20 employees who have similar backgrounds, experience, term of service, evaluation scores, etc.  7 of the 10 males received promotions, but only 4 of the 10 females received promotions.  This is the whole population of interest, but we can ask questions about the process.  Was the probability of promotion higher for males than females? or was it the same and random chance can explain the difference seen.  We could do a permutation test where we randomly assign 11 promotions to employees without considering sex, repeat a bunch of times, and see how often we have as large a difference or larger between the 2 groups due to chance.  This focuses on the process and gives reasonable results (we could also do a Fisher's exact test which is a shorthand way of evaluating all possible combinations for a simple case like this).

*No, it would not be wrong.  Cross-validation is a meaningful way to asses/evaluate (but not strictly test) random processes (including, but not limited to, sampling from populations).  So if you are happy with the assumptions used in the cross-validation analyses then you can use the results to describe your confidence in the results based on the full data.
