Repeatedly split data in training (0.75) and test (0.25) for cross validation What kind of cross validation is it called when we randomly split the data into 0.75 training and 0.25 test data set. And this split is done 1000 times.
 A: This is frequently referred to as repeated training-test split, or leave-group-out cross-validation, or Monte Carlo cross-validation$^1$. Note that this should be done with more splits than regular CV (which would have e.g. 10 partitions) - so you doing it 1000 times is a good idea.
$^1$ See e.g. Applied Predictive Modeling by Kuhn and Johnson, Springer, 2013.
A: Think you are referring to ShuffleSplit.
Shuffle Split perform random permutation of given data set and splits the data to train and test sets based on the chosen proportions you provide it as input.
As opposed to any other CV method - Shuffle Split never assumes that the folds it produce will be different than one another, meaning samples can appear in more than one fold.
This is really useful in cases like yours - where you may want a large number of random folds - but have a finite data set that can't be divided into a 1000 different folds, in strategies like KFold.
A link to sklearn's implementation of Shuffle Split:
http://scikit-learn.org/stable/modules/generated/sklearn.cross_validation.ShuffleSplit.html
A: I've also seen it under the name of repeated set validation. (The difference between hold-out and set validation in that context being that set validation results were used like cross validation results as surrogate for a final model trained on all data whereas hold out was refering to using the test results for exactly the model that was tested.
