How does linear SVMs function in multi dimensional feature space? I'm not able to picture how a linear SVM can perform classification in more than 2 dimensions. Also, when to chose linear SVMs and kernel based SVMs? For example, my dataset has over 5000 instances with 4000 features.
Linear SVM always works in the exact same way, even if you can't mentally wrap your head around the geometrics. Humans are generally bad at reasoning in more than three dimensions, so don't let that worry you.
Imagine linear SVM in one dimension. This would look something like this:
+ + + + + | - - - -
where | is your decision boundary. A hyperplane in one dimension is a cutoff value. In two dimensions, you get a line. In three dimensions, you get a plane, ...
With 4000 features in input space, you probably don't benefit enough by mapping to a higher dimensional feature space (= use a kernel) to make it worth the extra computational expense. Hence, use a linear kernel.
In fact, always use the linear kernel first and see if you get satisfactory results. Generally, you can try a nonlinear kernel if and only if you don't get good results with a linear kernel.
I was even stuck at same question for sometime, then I reffered some of the websites through which I was able yo get the intution.
If you are not able to visualise in mind consider this..
For 1 dimensional dataset a point will be suitable to discriminate between different classes.
Increase a dimension up for a 2 dimensional plane of data a single line may discriminate it with/without applying kernel.
Another increase will be analogous to having data in three dimensions but hyperplane or seperating plane in 2 dimension as we know a plane is 2d.
According to following Wikipedia article's starting lines you may develop the intution.
The hyperplane is just one dimension less than data in order to separate the data points into multiple classes.
Then for 4000 feature space it will be nothing other than 3999 dimensional plane (plane in order to seperate) or simply collection of the points with 3999 dimensions in order to seperate the data points.