Dealing with imbalanced data-set and cross-validation I have a data set of brain tumours, 700 malignant, and 225 benign. And I want to build a classification model using SVM, to classify the tumours types based on the data I have. My first question, is it considered an imbalanced dataset? if so, should I do undersampling of the malignant class?
Also, is it correct to use the below code to do cross-validation for my dataset? NOTE: groups = instances' labels vector (sorted malignant 0s then benign 1s) data = instances' data feature matrix
k=10;
cp = classperf(groups); 
cvFolds = crossvalind('Kfold', groups, k);   
for i = 1:k                                 
 testIdx = (cvFolds == i);                %# get indices of test instances
 trainIdx = ~testIdx;                     %# get indices training instances
 svmModel = fitcsvm(data(trainIdx,:), groups(trainIdx), 
'Standardize',true,'KernelFunction','RBF','KernelScale','auto');
 pred = predict(svmModel, meas(testIdx,:));
 cp = classperf(cp, pred, testIdx);

end
I still couldn't understand how crossvalind works? I mean does it guarantee that it takes instances from both classes at each fold?
 A: The fact that you are bringing up the issue of balance means that you have not considered the fact that proportion "classified" "correctly" is a discontinuous improper accuracy scoring rule.  If you use a proper scoring rule (e.g., Brier score or pseudo $R^2$) the issue goes away.  See this and this for more.
A: My first question, is it considered an imbalanced dataset?
Yes - there are almost three times the number of malignant to benign so you can consider this to be unbalanced. Generally, an unbalanced dataset will result in a model biased towards the class with most data.
If so, should I do under-sampling of the malignant class?
There are different approaches each with advantages and disadvantages. The main problem with undersampling is that you can lose information from the samples left out. With oversampling you create additional samples from the benign class – lets say you randomly duplicate existing samples. However, the training and test data is no longer independent so the issue here is that you can end up overfitting the model and all that implies – eg overestimating the model’s performance.
Cross Validation
Whichever approach you decide on you can mitigate the effects somewhat by performing cross validation, where the under or oversampling is performed on each fold. 
There is a lot of material online that can help – this link for example covers under and oversampling, as well as the option of ignoring the lack of balance and the implications of these options. It also covers SVM and Cross validation with example code in Python. This  should help you understand how to properly use cross validation.
