0
$\begingroup$

I am training an SVM model for the classification of the variable V19 within my dataset.

I have done a pre-processing of the data, in particular I have used MICE to impute some missing data. Anyway a part of the training dataset I use is this one:

training-set

Through the "tune" function I tried to train looking for the best parameters through cross-validation;

tune.out <- tune(svm, hard~., data=train, kernel="sigmoid",type="C",decision.values =TRUE,scaled =TRUE, ranges=list(cost=2^(-3:2),gamma=2^(-25:1),coef0=1^(-15:5)),tunecontrol = tune.control(nrepeat = 5, sampling = "cross", cross = 5))

I tried many combinations of parameters and different kernels, but what I get is always a model that can not predict correctly even the same training data, always returns all outputs to FALSE.

I really don't know if it's just a problem of tuning parameters or if I managed wrong the dataset.

Thanks for any advice.

EDIT : @Alex H I tried your code and what I obtain is :

Support Vector Machines with Radial Basis Function Kernel 
1094 samples
  18 predictor
   2 classes: 'X1', 'X2' 

Pre-processing: centered (18), scaled (18) 
Resampling: Cross-Validated (10 fold, repeated 5 times) 
Summary of sample sizes: 985, 985, 985, 985, 984, 984, ... 
Resampling results across tuning parameters:

  C     ROC        Sens    Spec
  0.25  0.5241539  0.9996  0   
  0.50  0.5320540  1.0000  0   
  1.00  0.5066151  0.9994  0   
  2.00  0.5225485  1.0000  0   
  4.00  0.5130391  1.0000  0   

Tuning parameter 'sigma' was held constant at a value of 0.04595822
ROC was used to select the optimal model using the largest value.
The final values used for the model were sigma = 0.04595822 and C = 0.5.
$\endgroup$
0
$\begingroup$

Try using the caret package.

library(caret)
set.seed(12345)

#Create simulation data
topxdata = matrix(rnorm(200, mean=0, sd=1), nrow = 20, ncol = 10)
botxdata = matrix(rnorm(200, mean=1, sd=1), nrow = 20, ncol = 10)
xdata = rbind(topxdata, botxdata)
colnames(xdata) = 1:10

ydata = c(rep("Top", 20), rep("Bottom", 20) )
ydata = as.factor(ydata)


# Setup for cross validation
ctrl <- trainControl(method="repeatedcv",   # 10fold cross validation
                     repeats=5,         # do 5 repetitions of cv
                     summaryFunction=twoClassSummary,   # Use AUC to pick the best model
                     classProbs=TRUE)


#Train and Tune the SVM
svm.tune <- train(x=xdata,
                  y= ydata,
                  method = "svmRadial",   # Radial kernel
                  tuneLength = 5,                   # 5 values of the cost function
                  preProc = c("center","scale"),  # Center and scale data
                  metric="ROC",
                  trControl=ctrl)

svm.tune

Post your result

Support Vector Machines with Radial Basis Function Kernel 

40 samples
10 predictors
 2 classes: 'Bottom', 'Top' 

Pre-processing: centered (10), scaled (10) 
Resampling: Cross-Validated (10 fold, repeated 5 times) 
Summary of sample sizes: 36, 36, 36, 36, 36, 36, ... 
Resampling results across tuning parameters:

  C     ROC    Sens  Spec
  0.25  0.980  0.85  0.91
  0.50  0.975  0.85  0.90
  1.00  0.955  0.83  0.88
  2.00  0.945  0.82  0.84
  4.00  0.945  0.81  0.77

Tuning parameter 'sigma' was held constant at a value of 0.06064355
ROC was used to select the optimal model using the largest value.
The final values used for the model were sigma = 0.06064355 and C = 0.25.
```
$\endgroup$
  • $\begingroup$ Thanks for the advice. $\endgroup$ – Marts Aug 16 at 16:32
  • $\begingroup$ Even with this solution I always obtain FALSE as prediction $\endgroup$ – Marts Aug 16 at 16:36
  • $\begingroup$ @Marts your ROC values are basically 0.5, and your specificity=0. This means the classifier isn't able to distinguish the two classes. Try a different classifier (e.g. change to method = "rf"). If that doesn't work you might have to collect more data. $\endgroup$ – Alex H Aug 16 at 17:44
  • $\begingroup$ thanks. I tried svmRadialWeights and I obtain some results, now I'm playing with parameters. I have to use SVM for this homework, maybe I will try sigmoid kernel, but in Caret I didn't find it as method. $\endgroup$ – Marts Aug 17 at 9:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.