1
$\begingroup$

I am following the steps outlined in this tutorial.

I have followed along and running into an issue at step 5.3. The output of the LDA model gives me all the expected information, except the Accuracy SD and Kappa SD are omitted.

Here is the code I ran and the results I got. I followed all steps of the tutorial, but left out some of the irrelevant code here.

> library(caret)   

> data(iris)
> dataset <- iris

> validation_index <- createDataPartition(dataset$Species, p=0.80, list=FALSE)
> validation <- dataset[-validation_index,]
> dataset <- dataset[validation_index,]

> control <- trainControl(method="cv", number = 10)
> metric <- "Accuracy"
> # a) linear algorithms
> set.seed(7)
> fit.lda <- train(Species~., data=dataset, method="lda",metric=metric, trControl=control)
> # b) nonlinear algorithms
> # CART
> set.seed(7)
> fit.cart <- train(Species~., data=dataset, method="rpart",metric=metric, trControl=control)
> # kNN
> set.seed(7)
> fit.knn <- train(Species~., data=dataset, method="knn",metric=metric, trControl=control)
> # c) advanced algorithms
> # SVM
> set.seed(7)
> fit.svm <- train(Species~., data=dataset, method="svmRadial",metric=metric, trControl=control)
> # Random Forest
> set.seed(7)
> fit.rf <- train(Species~., data=dataset, method="rf",metric=metric, trControl=control)
> results <- resamples(list(lda=fit.lda, cart=fit.cart, knn=fit.knn, svm=fit.svm, rf=fit.rf))
> summary(results)

Call:
summary.resamples(object = results)

Models: lda, cart, knn, svm, rf 
Number of resamples: 10 

Accuracy 
          Min.   1st Qu.    Median      Mean   3rd Qu. Max. NA's
lda  0.9166667 0.9375000 1.0000000 0.9750000 1.0000000    1    0
cart 0.7500000 0.8333333 0.9166667 0.9000000 0.9791667    1    0
knn  0.8333333 0.9166667 0.9583333 0.9416667 1.0000000    1    0
svm  0.8333333 0.9166667 0.9166667 0.9416667 1.0000000    1    0
rf   0.7500000 0.9166667 0.9166667 0.9250000 1.0000000    1    0

Kappa 
      Min. 1st Qu. Median   Mean 3rd Qu. Max. NA's
lda  0.875 0.90625 1.0000 0.9625 1.00000    1    0
cart 0.625 0.75000 0.8750 0.8500 0.96875    1    0
knn  0.750 0.87500 0.9375 0.9125 1.00000    1    0
svm  0.750 0.87500 0.8750 0.9125 1.00000    1    0
rf   0.625 0.87500 0.8750 0.8875 1.00000    1    0

> print(fit.lda)
Linear Discriminant Analysis 

120 samples
  4 predictor
  3 classes: 'setosa', 'versicolor', 'virginica' 

No pre-processing
Resampling: Cross-Validated (10 fold) 
Summary of sample sizes: 108, 108, 108, 108, 108, 108, ... 
Resampling results:

  Accuracy  Kappa 
  0.975     0.9625

I tried all of this again with the trainControl function altered to be:

 > control <- trainControl(method="repeatedcv", number = 10, repeat=3)

These results were identical to my first pass. Does anyone know why the model isn't giving the Standard deviation?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.