Kernel selection using SVM for keyword frequency classification

Question

I have data in Weka .arff multiple-class training and testing data representing daily word frequencies in RSS feeds as follows:

@relation _dm_19040_031925_06112013_1383748052958_Boolean-weka.filters.unsupervised.attribute.NumericToNominal-R193

@attribute Keyword_us_invest_are_Frequency numeric
@attribute Keyword_syrian_forc_kill_Frequency numeric
@attribute Keyword_europ_debt_crisi_Frequency numeric
@attribute Keyword_bank_of_america_Frequency numeric
@attribute Keyword_exclus_us_fugit_Frequency numeric
@attribute Keyword_debt_rate_cut_Frequency numeric
@attribute Keyword_on_debt_crisi_Frequency numeric
@attribute Keyword_market_fall_on_Frequency numeric
@attribute Keyword_russian_hockey_team_Frequency numeric
@attribute RSSFeedCategoryDescription {'Business and finance and economics','News and current affairs','Science and nature and technology',Sport,'Entertainment and arts'}

@data
0,0,0,0,0,4,0,0,0,0,0,0,0,0,0,0,7,0,0,0,0,0,0,0,0,0,0,0,0,0,0,4,4,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'Business and finance and economics'
0,3,0,4,0,0,2,1,0,1,2,0,0,0,0,0,1,0,0,1,3,2,0,0,0,0,0,0,0,25,0,0,0,0,1,0,0,2,0,0,1,1,2,1,0,0,0,0,2,1,0,2,0,2,1,2,4,0,2,0,0,0,0,2,0,0,0,0,0,0,0,1,0,0,0,0,0,0,1,1,0,0,0,0,0,0,0,0,0,3,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'Business and finance and economics'
0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,0,3,0,0,0,8,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'Business and finance and economics'
0,0,2,0,0,7,0,2,1,1,0,0,1,0,0,1,2,0,0,3,4,0,0,0,1,2,0,0,0,2,0,0,0,1,0,0,0,0,0,0,2,0,0,1,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,1,2,0,0,0,0,0,0,0,0,0,0,2,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,4,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,'Business and finance and economics'

There are up to 1500 rows of data and each row can be up to 192 columns long depending upon use of ngrams and frequency thresholds, stemming or stop words.

I am required to use Weka's LibSVM as a classifier upon this data which has five distinct classes, where the aim is to ideally produce comparable results to other classifiers in the 70 - 80% range.

Can anyone tell me which SVM kernel is best to use here?

I am not concerned with maximising accuracy here amongst the classifiers, just a ball-point figure will suffice.

Answer 1

Typically, for text represented as TF/IDFs you will use simple linear kernel since the dimensionality is high and there is no need to go even higher with a non-linear kernel.

Why do you bound the dimensionality to 192 features? It makes no sense to bound the dimensionality with a fixed set of features and then use non-linear kernels in an effort to artificially increase it.

If you do need to restrict dimensionality (e.g. for performance reasons) then the state of the art is to use a hashing trick to bound it.

Answer 2

There is no kernel that outperforms all other for all problems, even when the class of problems is reduced to a particular domain. Linear SVM are a popular choice for text classification, because many text classification problems are linearly separable, and because they scale well to large amount of data.

Still, you may try the three of them (linear, polygonal and radial basis function) and see which one performs better. The additional advantage of linear SVM is that there is only one parameter to tune (the regularization term). For polygonal and RBF you need to perform grid search (search over combinations of values for the regularization term and the rest of parameters, like the degree of the polynomial, for example).

Answer 3

In general the 'RBF' kernel will give the best results. However, it does suffer when dealing with large datasets so I would first try 'RBF' and if that is taking to long try switching to a linear kernel. HTH.