Machine learning in really small dataset The group that I work with sent me 10 samples, 5 controls and 5 treatment. They asked me to try to use machine learning in this dataset, to see if it could separate the two groups.
I believe that this amount of data is very very low, and I'm quite sure that I can't get any reliable result from this. But, as a beginner in Machine Learning, I don't have the knowledge to explain to them why it is a complicated thing to do.
So, in order to understand better the problem here, my questions are:

*

*Is it is even possible to train a model using only 10 samples?


*If possible, what is the appropriate cross-validation approach to use in such small dataset?


*In order to explain to the group, why is it hard to get reliable results from small datasets?
Also, I'm trying to read as many articles as possible about ML, but the content is vast and I'm quite lost. Any recommendations of articles that discuss these problems that I pointed are very welcome.
 A: You can train a partition tree on ten data points even with an arbitrary amount of features and run LOOCV over that: If there is a very simple and nearly 100% reliable distinction (feature no 3 is 1 in group A and 2 in group B) than a tree could find that. If there is some complicated rule as distinction, you will not find that in n = 2 x 5.
Can you train a model? Yes, you can. The following will grow a randomForest on n = 2*5 with only two features. Is it sensible to train a randomForest on only two predictors? Who cares. On these obvious data it will compute an accuracy and correctly tell you, which of the two predictors contains the relevant information:
library(caret)

expl <- data.frame(group = gl(2, 5),
                   a = c(1, 2, 3, 4, 5, 10, 11, 12, 13, 14),
                   b = rnorm(10))

f <-train(group ~ ., 
          data = expl,
          method ="rf",
          trControl = trainControl(method = "LOOCV"),
          verbose = FALSE,
          importance = TRUE)
f

varImp(f)

Sometimes it is simple to get more observations, sometimes it's hard. If there is only very little information, you should first check, whether there is useful prior information for falling back to Bayes statistics. If not you need to decide whether it's best to say nothing at all based on weak evidence or whether it is worthwhile to use every little bit of information there is. Sometimes the latter is worthwhile even if unsatisfactory.
