Does anybody know in which cases --of learning and predicting--, it is better to use "validation test" or something else, instead of "k-fold cross validation" to assess the performance of the algorithm in predicting the target classes? Here is my case and problem: Yes, I know that if the data is less, CV works well, but this case --with limited samples-- is a bit different. Because the order of the data as the test set is important for us, so that we can not apply CV, since CV gives you the average performance of all folds. More in detail: I have the following data set with 4 features and 10 samples;
f1,f2,f3,f4,class S1 S2 S3 S4 S6 S7 S8 S9 S10
Every sample referees to an activity that accrued in a specific date (E.g, S1 logged in the first week, S2 logged in the second week, and so on). It means an activity that logged in sample (S1) is much different from the last sample (S10), on the other-hand S1 and S2 or S3 are not much different. So in this case, I used S1 to S8 as the training set, and S9 and S10 as the test set to validate the classification performance. Since, the samples and values change over time, I could not use CV to validate the performance. So my questions is, can I rely on this statement and my way to design the training and test is reliable?