Using repeat measurements to increase validity? User Dave posted an answer to a relatively simple question, but with an interesting twist in the comments. Here, the idea seems to be to run repeat measurements on random samples, drafting a histogram based on p-values obtained.
I'm wondering if this is the same as k-folding, or if it's a different approach? I'm looking for a general explanation of the application and reasoning for using this method, and does it increase the validity of a study? If not, what boons does it achieve as compared to simply running one regression/hypothesis test on the dataset and obtaining one p-value?
 A: The task behind the question to which you link, for some unspecified reason, requires repeated subsampling of a random 15% of a data sample. That's throwing away 85% of the data each time. In general, throwing away data isn't a good idea. If you have 240,000 data points, why not just use them all at once?
The procedure is not $k$-fold cross-validation. That means breaking apart the total data sample into $k$ non-overlapping subsets, building $k$ models on data from which one subset is held out, evaluating the model on the held-out subset, and combining the evaluations over all $k$ models to get an estimate of modeling performance. Nor is it bootstrapping, in which you generally take random samples with replacement of the same size as the original sample to try to evaluate the distribution of a statistic in the underlying population. In the linked question, there is no obvious purpose to the repeated 15% sub-sampling except to get a large set of p-values based on less-than-complete data.
My sense is that the basis for both the original task and Dave's  suggestion is pedagogical. The idea is presumably to treat the 240,000 data points as a complete population, with each 15% representing a sample from the population. If the null hypothesis holds, then among the multiple samples there should be a uniform distribution of p-values over [0,1]. If the null hypothesis does not hold, then the p-values will be more concentrated toward 0. Examining the distribution of p-values in the latter case will illustrate the power of the test (the fraction of samples in which the p-values are less than the threshold for "significance). Simply repeating the same analysis on multiple subsets of a large data set, as in the question to which you link, doesn't help the statistical analysis except for that pedagogical purpose.
