My background is in ecology, it is common to have smaller sample sizes and class imbalances and ML approaches are still increasingly adopted. My specific dataset: training set is 49 sample, my test set is 16 samples - total 65 samples. I have also used loocv (not reported in the table) where I observed similar pattern.
My class proportions are as such across 6 classes, when I stratify it 75-25%, it is to make sure class 5 (rare class) is represented in training-test split and stratified k-folds as well for loocv and 5-fold cv:
class 1: 19 samples
class 2: 9 samples
class 3: 9 samples
class 4: 15 samples
class 5: 5 samples
class 6: 8 samples
I know accuracy is low in general - I have attempted to optimize it better using different variable selection methods, etc. I understand its not suitable for ML in general but technically, I was curious what the issue could be - just the size of samples, should stratification be done different for training-test split, etc.
All analysis in R programming environment.
Thank you in advance.