# Uncertainty in Enumerated Data

I am interested in assessing possible uncertainty in enumerated data.

To do this, I randomly pick 80% of the enumerated data to predict the remaining 20% through a regression analysis. I reapeat the process 1000 times. Ideally, this should allow me to assess uncertainty affecting explanatory and predictive performances in a regression context.

Is that correct?

• In permutation test you sample without replacement $N$ values out of sample consisting of $N$ cases (i.e. randomly shuffle the cases) and then compute some statistic $S$ on such sample. This enables you to generate null distribution of $S$ so to compute probability of observing such (or more extreme) value of $S$ as you obtained on your original sample.
• In bootstrap you sample with replacement $N$ values out or your original sample. This lets you to re-create by simulation distribution of your sample and asses uncertainty of your estimate.
• In cross-validation you randomly pick some proportion $p$ of $N$ cases, so to use $p \times N$ cases for training your model and $(1-p) \times N$ cases for prediction. By this you can learn about uncertainty of your predictions.