The title doesn't make sense but allow me to explain. I have a set of gene expression data with over 10000 genes as features. There are roughly 30 samples, and let's say there are 10 from class A, 10 from class B and 10 others. In the "others" samples, each of them has a potentially distinct class, but may also be in class A, B or a (few) new class(es) C(,D,E,etc.). My hypothesis is that if I can select a set of feature for A, B and each of the samples in "others", then do a clustering such as DBSCAN/OPTICS, I can reveal the structure of the data set, putting the "others" into A, B, some new classes, or an outlier that doesn't belong to any class.

I have tried information gain in FSelector in R with two different filtering methods: (1) doing two separate information gain calculation for class A vs B+others and class B vs A+others; (2) doing one information gain calculation by labelling the samples A, B and as individual classes for all samples in "others" i.e. each class label for each "others" sample contains one and only one sample.

Which one is the correct way/better way? And is there any good suggestions for this type of feature selection with single cases in multiple classes e.g. svm,lasso?

  • $\begingroup$ Why not clustering? $\endgroup$ – user2974951 Aug 6 at 7:31
  • $\begingroup$ Would you mind elaborate a bit? Do you mean clustering features for feature selection? For semi-supervised feature selection I actually tried this paper but I feel like the supervised part of the process will mask the results from the unsupervised part. $\endgroup$ – Kent Aug 6 at 15:59

The problem, as stated, is a little weird. You say your sample has classes A, B and others. But you are unsure about the others as they may belong in A, B or something else completely?

You need to decide what you are going to do here, are you going to assume the classes are known and do supervised learning? Are you going to assume the classes are not really known and do unsupervised learning? Or maybe something in the middle like reinforcement learning?

I would suggest unsupervised learning in your case, if you are not sure about the classes. But first, 10000 variables is a lot of variables, you will get suboptimal performance with these many variables and your sample size, you may consider removing some variables in a pre-filtering stage based on some criteria.

As for the model, something like kmeans / pam / klara could work, where you would determine the number of clusters by using one of the measures for this, such as silhouette, gap statistic, wss.

Also, for feature selection, something like NSC (Nearest Shrunken Centroids) will work, which automatically selects features.

  • $\begingroup$ I will have a look at NSC. However, I feel like I am losing a lot of information when I perform a fully unsupervised feature selection, given that I do know A and B and I actually know one of the "others" is not in A and B (somewhat like an outgroup so that I can monitor the clustering). My plan was on the line of doing a semi-supervised feature selection and then an unsupervised learning. OPTICS was my choice because of the nature of my data, which is RNA-Seq data from patients with class A and class B potentially having a different density. $\endgroup$ – Kent Aug 7 at 11:40
  • $\begingroup$ The genomic context for every one in the "others" is different, which is why I want to get some features that are specific for them but not others. $\endgroup$ – Kent Aug 7 at 11:44
  • $\begingroup$ Oh sorry I just checked NSC is supervised. I will see if I can incorperate it with some unsupervised feature selection method. $\endgroup$ – Kent Aug 7 at 14:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.