I am working with ML models where the data is plentiful: I can get billions of records for training and validating my models. In fact, I have so much data, I need to sample to reduce the data set sizes (to a few tens of millions of records).
In that context, do I need to strive to do the splitting and sampling "right", i.e. splitting first then sampling with replacement for the training and validation data sets? It seems that even if I sample without replacement, then split, given the abundance of unique data points, I will be "close enough" to being i.i.d. and the risk of leaking the validation set in the training would be negligible?
(In my case there is a performance difference between sampling with and without replacement, and I would like to avoid the sampling with replacement, if safe enough)