I am trying to use sequentialfs to do some feature selection in matlab. I have huge dimensional data of 22215 features. When I tried to use sequentialfs with svm as classifier so that it selects the best subset of features, it just keeps on running, probably its because of the huge number of dimensions. However, weka does the same thing very quickly. Weka has wrapper filters and it does it so quickly. What sort of heuristic does it use? Even though I tried with 5000 features in weka since it was not taking 22215 features, it gave me results quickly with wrapper filters. What should I do with sequentialfs in matlab

This is the command I am using in matlab

c = cvpartition(yS1,'k',12);
opts = statset('display','iter');
[fs, history] = sequentialfs(@SVM_class_fun, X, yS1,'cv', c, 'options', opts);


function err = SVM_class_fun(xTrain, yTrain, xTest, yTest)
  model = svmtrain(xTrain, yTrain);
  err = sum(svmclassify(model, xTest) ~= yTest);

Here the dimension of X is 100x22215 where I have 100 examples each of dimension 22215.

  • $\begingroup$ Which classifier are using in the wrapper FS of WEKA? Probably, WEKA is not using SVM which takes time to train. $\endgroup$
    – soufanom
    Mar 23, 2013 at 11:13
  • $\begingroup$ @soufanom. I am not planning to use weka. I need to do that in matlab. Sequentialfs didn't work in matlab. Am I doing something wrong or it's just not possible using sequentialfs. I need to use some heuristic to prevent searching for all possible features, I gues $\endgroup$
    – user34790
    Mar 23, 2013 at 13:56
  • $\begingroup$ Are you using the default setting of sequentialfs in Matlab including 'direction' option? It might be more appropriate to share your Matlab command. $\endgroup$
    – soufanom
    Mar 23, 2013 at 14:08
  • $\begingroup$ @soufanom. I have shared the code. I am currently using SVM, but I am fine with any other classifier. $\endgroup$
    – user34790
    Mar 23, 2013 at 14:23

3 Answers 3


Can you not make MatLab call the WEKA and pass data off to it?

For very high dimensional data I find the work of Eugene Tuv to be very useful. Random forests of Gradient Boosted Trees can whittle 100k rows and 100k columns to something useful in a very short time.



That's a lot of features to input into a model. Have you thought about redundancy of information within the features? I would suggest dimensional reduction using e.g. PCA or non-linear manifold learning (diffusion maps, Laplacian eigenmaps, locally preserving projections, Sammon mapping, or even SOM, etc.). Would not some of your features also be noisy with little informativeness?


The problem is that you have a very large number of features as compared to that of the observations. There might be many redundant features, you should first check for that, and remove the features based on the rankings obtained. You can compare the correlation among the features (this should be minimised) as well as between the feature and label (this should be maximised), this will provide a clear idea of which are relevant features and which can be got rid of. (Refer to minimum redundancy maximum relevance algorithm for this - Ranking of features using MRMR algorithm)

After reducing the number of features, apply sequential feature selection (sequentialfs() function in MATLAB, here you can avoid SVM and choose LDA to reduce the computational time further).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.