I have a dataset consisting of 5 subjects (5 different virus names and 5 benign program names). The programs belong to the class "benign" and the viruses to the class "infected". There are 1000 instances in the dataset, each owning several attributes (besides class and subject).
I now want to conduct an experiment where I take this dataset and measure the performance of different classifiers regarding unknown benign and infected instances.
In order to do that I want to run a leave-one-out cross validation: in 5 runs a classifier shall be trained by 4 subjects of class benign and 4 subjects of class infected. Afterwards the classifier shall be tested on the remaining subject of class benign and the one from class infected. This I want to repeat 5 times with changing test subjects so that in the end every subject was a test-subject one time.
I want to do this without preparing my datasets by hand. is there a way in weka 3.8 to automate this task in the knowledge flow?
Thanks in advance!