# When we should NOT use k-fold cross validation to assess the predictor?

Does anybody know in which cases --of learning and predicting--, it is better to use "validation test" or something else, instead of "k-fold cross validation" to assess the performance of the algorithm in predicting the target classes? Here is my case and problem: Yes, I know that if the data is less, CV works well, but this case --with limited samples-- is a bit different. Because the order of the data as the test set is important for us, so that we can not apply CV, since CV gives you the average performance of all folds. More in detail: I have the following data set with 4 features and 10 samples;

            f1,f2,f3,f4,class
S1
S2
S3
S4
S6
S7
S8
S9
S10


Every sample referees to an activity that accrued in a specific date (E.g, S1 logged in the first week, S2 logged in the second week, and so on). It means an activity that logged in sample (S1) is much different from the last sample (S10), on the other-hand S1 and S2 or S3 are not much different. So in this case, I used S1 to S8 as the training set, and S9 and S10 as the test set to validate the classification performance. Since, the samples and values change over time, I could not use CV to validate the performance. So my questions is, can I rely on this statement and my way to design the training and test is reliable?

• By "validation test" do you mean measuring performance a single, held-out validation set rather than repeated over k folds? – user20160 Sep 21 '18 at 10:40
• Yes, E.g., 70% of the data as training and 30% as the test set, considering if we have around 10 samples --of course they are few--. – nahid khosh Sep 21 '18 at 10:45
• A test set is preferred over CV, when possible. So if you have a big enough data set and can afford to split your data then you should do so, otherwise, if you do not have a large enough sample, CV works well. – user2974951 Sep 21 '18 at 10:49
• essentially its a computational issue. if it takes too long to train you use single split of data (eg enormous Neural networks). Equally if its too long because you have so much data, then a single split will still be reliable. [andrew ng makes this statement in coursera.org/learn/deep-neural-network] – seanv507 Sep 21 '18 at 10:50
• Yes, I agree that if the data is less, CV works well, but this case --with limited samples-- is a bit different. I have updated my question with more detailed information. – nahid khosh Sep 21 '18 at 12:36