Usually, when I am given a dataset of $d\le 3$, I just plot the data and observe if linearly separable behaviour exists. When the dataset is of high dimensionality, I always follow a simple trick to identify if it is linearly separable or not: I run an SVM classifier using Linear kernel, and if the resulting accuracy is high (more than 90%), then I can say this dataset is linearly separable. If not, then I know this dataset is not linearly separable so I go to sophisticated ML algorithms like neural networks. Do you think this is a good approach?
$\begingroup$
$\endgroup$
1
-
1$\begingroup$ That sounds OK. In a way, svm's was made just to that job ... $\endgroup$– kjetil b halvorsen ♦Commented Nov 1, 2017 at 16:10
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
That approach sounds right, another approach is to try to reduce the dimensions of the dataset and observe if the data is separable in the lower dimension space (via plots) there are different algorithms for doing this and they all maintain different structural properties, but the commonly used approach is the PCA and LDA. Other methods exists such as the MDS which maintains the global distances.
$\begingroup$
$\endgroup$
In practice, data sets are rarely linearly separable.
In fact, it is recommended by the developers of LIBSVM to proceed right to training an SVM using a Gaussian kernel, experimenting with different bandwidths.
The advantage of SVM's over NN's is that SVM's are more interpretable, so if you're going to end up explaining your result to perhaps the issuer of the data, then this approach could work well.
I would say that in most cases, NN's will do a better job of classification, provided they are structured optimally.
