# Hold out validation. Exactly what is left out?

I'm studying validation and I've seen multiple examples of hold out validation. Some will hold the tail of the data, while others will leave out $$n$$ random points.

I assume it has to do with whether you're testing for interpolation or extrapolation, but is there something I'm missing as far as general practice goes? Because when you just leave out random points it just seems like 1 step of k-fold validation, without training and validating over the rest of the folds.

Thanks!

• Selecting the tail of data is a terrible idea if your data is sorted in some systematic way that affects prediction. For example, if your data is sorted by date and you are modeling something that might be affected by date, experience, etc., holding out the most recent records might paint a worse (or better) picture of your model than selecting random records. – StatsStudent Jan 11 at 18:57
• @StatsStudent Holding out the "tail" if they mean the most recent records (I would not call this tail though..) might make sense if you are comparing models intended to test on future data. In that case, testing against the most recent data is more like the real-world situation and might help you know if it makes sense to use a model that favors more recent data in fitting the model parameters. I agree in most cases it is a bad idea, though, just not necessarily terrible in every case. – Bryan Krause Jan 11 at 19:30
• Agreed 100% Bryan. I think Yilun Zhang's answer below addresses this aspect well. – StatsStudent Jan 11 at 19:36

• @George I think what I meant is that it's not reasonable to make a prediction for a time point in the middle of two training data points for time series data since the value can possibly be inferred from a future time point. Moreover, if you already have data point for $t_{2018}$, then why do you want to make a prediction for $t_{2017}$ ? – Yilun Zhang Jan 12 at 22:20