# 10 fold cross validation model in weka

I'm trying to build a specific neural network architecture and testing it using 10 fold cross validation of a dataset. Now building the model is a tedious job and Weka expects me to make it 10 times for each of the 10 folds.

Can't I just make the model for the first fold and ask weka to use that same model for the remaining 9?

Also, since building neural networks in weka is so easy, can I import a neural network structure to matlab for further use?

• 10-fold CV consists of building the model 10-times. There is no way around that. After all, in each fold you will have different train set. – alesc Apr 25 '15 at 19:43
• I think you need to read up on what cross-validation does. Based on your question, it appears you don't know what cross-validation is for. – Marc Claesen Aug 27 '15 at 18:08
• Chapter 5 of An Introduction to Statistical Learning provides a good intro to cross validation. – Tchotchke Aug 27 '15 at 19:24
• Could you tell us why you would want to use multi fold cross validation instead of a simpler train test split (1 fold). Since neural networks take longer to train, train test splits are used more often than n fold validation. – shark8me Apr 27 '16 at 7:31

You can use the Evaluate class to perform this 10-fold cross-validation. To define the cross-validation you have to set the parameter as '-x 10' in the EvaluateModel.

clear all; close all; clc;

%% Add jar file to path plus import dependencies
import weka.classifiers.trees.RandomForest.*;
import weka.classifiers.meta.Bagging.*;
import weka.classifiers.Evaluation.*;
import weka.core.Instances.*

%% load the arff file and extract the informations
filename = 'algo_output/results_features_labeling2_2class.arff';
data.setClassIndex(D.numAttributes()-1);

%% classification
classifier = weka.classifiers.functions.MultilayerPerceptron();
classifier.buildClassifier(data);
classifier.toString()

%% 10-fold cross-validation
ev = weka.classifiers.Evaluation(data);

v(1) = java.lang.String('-t');
v(2) = java.lang.String(filename);
v(3) = java.lang.String('-x');
v(4) = java.lang.String('10');
v(5) = java.lang.String('-i');

prm = cat(1,v(1:end));
ev.evaluateModel(classifier, prm)


• Just a remark: Weka's internal algorithm for performing $k$-fold CV is single-threaded. Since you are dealing with Neural Networks, which take a very long to train, you will desperately need multi-threading. If you need a Java implementation of a multi-threaded $k$-fold CV, I can provide you the code. – alesc Apr 25 '15 at 19:47
• If you have small samples, then even MultilayerPerceptron will finish quickly. I have been using large datasets that can take days to train, and without multi-threading, it would take much longer. Here is my code for benchmarking: pastebin. I have listed the source code for 3 files, but the first file is the most important. Since I am using Streams and lambda expressions, you will need Java 8 to run it. – alesc Apr 26 '15 at 8:03