1
$\begingroup$

I have a dataset consisting of 5 subjects (5 different virus names and 5 benign program names). The programs belong to the class "benign" and the viruses to the class "infected". There are 1000 instances in the dataset, each owning several attributes (besides class and subject).

I now want to conduct an experiment where I take this dataset and measure the performance of different classifiers regarding unknown benign and infected instances.

In order to do that I want to run a leave-one-out cross validation: in 5 runs a classifier shall be trained by 4 subjects of class benign and 4 subjects of class infected. Afterwards the classifier shall be tested on the remaining subject of class benign and the one from class infected. This I want to repeat 5 times with changing test subjects so that in the end every subject was a test-subject one time.

I want to do this without preparing my datasets by hand. is there a way in weka 3.8 to automate this task in the knowledge flow?

Thanks in advance!

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

Welcome to the site! What you refer to is called a stratified cross-validation and, as you allude to, in limited data-sets a very good idea.

The developers / authors of Weka state "Weka does stratified cross-validation by default.[1]", which is a prudent design choice.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.