# Handle large set of features using SVM

I have a biological dataset with 30.000 features (genes) and 1000 data points (cells). Basically I have two major classes of cells: 1 and 0 with a distribution of 90/10.

Now I am trying to classify these correctly using nested cross validation. The first thing I tried was to manually decrease the number of features by considering biological relevant subsets of the total feature set (reduced to 20 features), which gives me reasonable results (0.7 F2 score).

However, I am wondering if I use the whole feature set if I will get big overfitting since I have much less data points than features.

Is it true that I would overfit my data if I use the whole feature set? And if so, are there any ways to decrease the feature set without prior biological knowledge?

Thanks a lot! Tomi

• – Dougal Aug 25 '14 at 8:52

That said, if you have more features you will likely need to regularize stronger since typically the training errors increase in size, which can induce an overfit (e.g. you probably need to use a lower $C$).
• You can equally well argue that it happens to compute instance weights as a byproduct of feature weights. In the primal, you optimize over $w$ and $b$; in the dual, over $\alpha$. The two are equivalent, and which you solve depends on what software you used. (Typically primal solvers are preferred if available and there are more instances than feature dimensions, but that's a computational issue irrelevant to the actual solution.) – Dougal Aug 25 '14 at 9:01