1
$\begingroup$

I have data in Weka .arff multiple-class training and testing data representing daily word frequencies in RSS feeds as follows:

@relation _dm_19040_031925_06112013_1383748052958_Boolean-weka.filters.unsupervised.attribute.NumericToNominal-R193

@attribute Keyword_us_invest_are_Frequency numeric
@attribute Keyword_syrian_forc_kill_Frequency numeric
@attribute Keyword_europ_debt_crisi_Frequency numeric
@attribute Keyword_bank_of_america_Frequency numeric
@attribute Keyword_exclus_us_fugit_Frequency numeric
@attribute Keyword_debt_rate_cut_Frequency numeric
@attribute Keyword_on_debt_crisi_Frequency numeric
@attribute Keyword_market_fall_on_Frequency numeric
@attribute Keyword_russian_hockey_team_Frequency numeric
@attribute RSSFeedCategoryDescription {'Business and finance and economics','News and current affairs','Science and nature and technology',Sport,'Entertainment and arts'}

@data
0,0,0,0,0,4,0,0,0,0,0,0,0,0,0,0,7,0,0,0,0,0,0,0,0,0,0,0,0,0,0,4,4,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'Business and finance and economics'
0,3,0,4,0,0,2,1,0,1,2,0,0,0,0,0,1,0,0,1,3,2,0,0,0,0,0,0,0,25,0,0,0,0,1,0,0,2,0,0,1,1,2,1,0,0,0,0,2,1,0,2,0,2,1,2,4,0,2,0,0,0,0,2,0,0,0,0,0,0,0,1,0,0,0,0,0,0,1,1,0,0,0,0,0,0,0,0,0,3,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'Business and finance and economics'
0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,0,3,0,0,0,8,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'Business and finance and economics'
0,0,2,0,0,7,0,2,1,1,0,0,1,0,0,1,2,0,0,3,4,0,0,0,1,2,0,0,0,2,0,0,0,1,0,0,0,0,0,0,2,0,0,1,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,1,2,0,0,0,0,0,0,0,0,0,0,2,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,4,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,'Business and finance and economics'

There are up to 1500 rows of data and each row can be up to 192 columns long depending upon use of ngrams and frequency thresholds, stemming or stop words.

I am required to use Weka's LibSVM as a classifier upon this data which has five distinct classes, where the aim is to ideally produce comparable results to other classifiers in the 70 - 80% range.

Can anyone tell me which SVM kernel is best to use here?

I am not concerned with maximising accuracy here amongst the classifiers, just a ball-point figure will suffice.

$\endgroup$

3 Answers 3

0
$\begingroup$

Typically, for text represented as TF/IDFs you will use simple linear kernel since the dimensionality is high and there is no need to go even higher with a non-linear kernel.

Why do you bound the dimensionality to 192 features? It makes no sense to bound the dimensionality with a fixed set of features and then use non-linear kernels in an effort to artificially increase it.

If you do need to restrict dimensionality (e.g. for performance reasons) then the state of the art is to use a hashing trick to bound it.

$\endgroup$
3
  • $\begingroup$ The feature list is limited to 192 deliberately because that is the maximum number of keywords used for the period in question. That is 64 unigrams, 64 bigrams and 64 trigrams. Each frequency is the raw frequency per word in a feed for a day. $\endgroup$
    – Mr Morgan
    Jul 21, 2014 at 17:14
  • $\begingroup$ OK, not sure why you do that, but my answer still stands. You can use TF/IDFs for bigrams and trigrams, and there is no point in using a non-linear kernel instead of increasing the 192 limit of yours. $\endgroup$
    – iliasfl
    Jul 21, 2014 at 23:27
  • $\begingroup$ The 192 limit is fixed but I still have to consider the kernel. I'm not concerned with maximising the result, I'm more concerned with matching SVM results to other classifiers which have run over the same data. Thanks for the input and I see that I need to consider a few things when choosing which kernel. $\endgroup$
    – Mr Morgan
    Jul 22, 2014 at 0:04
1
$\begingroup$

There is no kernel that outperforms all other for all problems, even when the class of problems is reduced to a particular domain. Linear SVM are a popular choice for text classification, because many text classification problems are linearly separable, and because they scale well to large amount of data.

Still, you may try the three of them (linear, polygonal and radial basis function) and see which one performs better. The additional advantage of linear SVM is that there is only one parameter to tune (the regularization term). For polygonal and RBF you need to perform grid search (search over combinations of values for the regularization term and the rest of parameters, like the degree of the polynomial, for example).

$\endgroup$
0
$\begingroup$

In general the 'RBF' kernel will give the best results. However, it does suffer when dealing with large datasets so I would first try 'RBF' and if that is taking to long try switching to a linear kernel. HTH.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.