Classifying 10 top features using SVM

Question

I'm new in R. I have a problem with classification using SVM. In my train_data, I have 82 samples and 25770 features while in my test_data I have 36 samples and 25770 features. The last column is the class-label. I would like to classify my samples into "normal/control/0" and "diabetes/1" using 10 top features/genes that I gained from Feature Selection. From feature selection, I have this:

And this is my code for classification:

training_dataset<-data.frame(train_data[,top.features$FeatureID],y=train_data[,ncol(train_data)])
training_dataset$y<-factor(training_dataset$y)
test_data$y<-factor(test_data$y)

svm_RBF<-function(class.dataset,c,gamma,test.data){
class.dataset$y=factor(class.dataset$y)
svm.RBF=svm(y~., data=class.dataset,cost=c,gamma=gamma,kernel='radial',types='C-classification')
predict.svm.RBF=predict(svm.RBF,test.data)
confusionMatrix.RBF=confusionMatrix(predict.svm.RBF,test.data$y)
print(confusionMatrix.RBF)
plot.confusionMatrix.RBF<-function(confusionMatrix.RBF){
mytitle<-paste("Accuracy", percent_format()(confusionMatrix.RBF$overall[1]), "dan Balanced Accuracy",
               percent_format()(confusionMatrix.RBF$byClass[8]))
p<-ggplot(data = as.data.frame(confusionMatrix.RBF$table),
          aes(x=Reference,y=Prediction))+
  geom_tile(aes(fill=log(Freq)),colour = "white")+
  scale_fill_gradient(low="white", high = "steelblue")+
  geom_text(aes(x=Reference, y=Prediction, label=Freq))+
  theme(legend.position = "none")+
  ggtitle(mytitle)
return(p)
}
print(plot.confusionMatrix.RBF(confusionMatrix.RBF))
print(roc.curve(test.data$y, predict.svm.RBF))
}

training_dataset10<-data.frame(training_dataset[,1:10],y=training_dataset[,ncol(training_dataset)])
svm_RBF_10<-svm_RBF(train_dataset10,32,0.001953125,test_data)

Now the problem is, the model that I just made have high accuracy but low in AUC. My questions are :

1. Is this the correct way to do classification using 10 top features?
2. Is this the correct way to plot my classifier?
3. Do you have a better suggestion to increase my AUC? The data already balanced.

I already did generalization error and parameter tuning so those are the best parameter that I get.

Answer 1

Firstly, why are you only using one classifier? See this answer on data mining theorems and problems with using one classifier. You should be using linear regression first, possibly followed by logistic regression, Naive Bayes, k-nearest neighbor, linear discriminant analysis, etc. You should also be considering use of an ensemble of classifiers based on majority committee voting.

You should also consider cross-validation.

Take a look at the different ROC-AUC curves from different classifiers here