I have experimental data of 25 human subjects. What is the efficient way to split it into train, test, and validation? It is given that,the number of data is different for different subjects. This means subject n1 might have a large volume of data where n2 might have few samples of data.
I am using four-fold validation with 5 subject data kept as testing and the rest of 20 as training and validation. The problem is splitting validation data from these 20 subjects. I can assume randomly take two or three participant data as validation. However, some participant has a large number of the data point. If anyway those participant is selected as validation then validation data will have a chance to greater or equal to the training set, which does not make sense.
Is there any better approach? or should I fix the validation subject for each fold?