1
$\begingroup$

I'm using the following data set on some credit scoring models: https://archive.ics.uci.edu/ml/datasets/Statlog+(German+Credit+Data)

My teacher told me that it's best to use the same data set for all the different techniques, but how do you handle the different restrictions?

The data set consists of 7 numeric and 13 categorical variables, how do you use those categorical variables for the sum? Doesn't the support vector machine only accept 1 or 0 as input? Or is it values ranging between 0 and 1 ?

$\endgroup$

2 Answers 2

3
$\begingroup$

Doesn't the support vector machine only accept 1 or 0 as input? Or is it values ranging between 0 and 1

No. SVMs accept continuous arguments. Categorical ones are the problem, which can be solved by using dummy variables. The inputs do not have to be bounded, but most beginner guides recommend normalizing all inputs to the interval $[0, 1]$. It is important to ensure that all inputs have comparable scales, whatever that may be.

$\endgroup$
6
  • $\begingroup$ so if you use dummy variables, and the variable count run up to the 60+, doesn't that increase the complexity and thus worsen the model? $\endgroup$ Commented Aug 10, 2014 at 19:41
  • $\begingroup$ 60 variables is no problem. SVMs can properly deal with thousands. For SVMs the number of variables is irrelevant, as the model is specified in terms of training instances (because of the kernel trick) instead of training variables. Standard linear regression models, in contrast, suffer from the curse of dimensionality, because they must estimate a coefficient for each variable. $\endgroup$ Commented Aug 10, 2014 at 20:39
  • $\begingroup$ so what would you do then if you would need to use the same training data on all different techniques (logistic regression, neural network, svm, decision tree and k-NN)? Would you change all categorical variables by dummy variables then? And use those same dummy variables for the other techniques? $\endgroup$ Commented Aug 11, 2014 at 7:43
  • $\begingroup$ @user3127227 the fairest approach would be to consider making dummy variables (when necessary) as part of a technique. Decision trees, for instance, can work with categorical data out of the box, which is a strength that deserves to be taken into account. $\endgroup$ Commented Aug 11, 2014 at 7:50
  • $\begingroup$ If i'm understanding the algorithms in weka correctly, there is no need to transform the variables as for example the SMO transforms nominal attributes into binary ones. $\endgroup$ Commented Aug 11, 2014 at 10:38
1
$\begingroup$

The best solution is to use OneHotEncoding for categorical attributes. Have a look at implementation of OneHotEncoding on sklearn on this link. http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.OneHotEncoder.html

After you convert these categorical data into OneHotEncoded data. It will be like for first attribute the values are A11, A12, A13, A14. so lets say we have training example that starts with A12. It will be converted into 0 1 0 0 in OneHotEncoding. You can similarly convert for all categorical attributes.

For numerical data, normalize them to zero mean and standard deviation of 0.01. Although this step is not compulsory, but machine learning algorithms perform good on the normalized data.

For the categorical attributes with two values, the best is to give them +1 and -1 Encoding.

Once you do this for training data, you can easily import using pandas and differentiate between training data (all columns except last) and training label(last column). Then you can train using sckikit learn.

I applied this strategy and for me RandomForestClassifier with 100 trees gave the best performance of 80% accuracy on 100 validation dataset that I separated from the training data.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.