Skip to main content
Bumped by Community user
Bumped by Community user
Bumped by Community user
Bumped by Community user
Bumped by Community user
Bumped by Community user
removed unnecessary words from title
Link
Frank Harrell
  • 98.5k
  • 6
  • 192
  • 448

How to determine Accuracy/Classification rate with 5-fold cv in training and test set? (specific case, look below)

Source Link

How to determine Accuracy/Classification rate with 5-fold cv in training and test set? (specific case, look below)

I'm trying to use 5-fold cv for training set and test set but I can't determine the confusion matrix using table() to calculate the classification rate (true positives and true negatives divided by total number) because the sets have different lengths but I'm probably doing it wrong:

(after defining dataset using read.table and column names)

#turn dataset lines into random order for splitting
variavel <- runif(nrow(dataset))
dataset <- dataset[order(variavel),]

#create 5 folds and set variable (vector) to save accuracy rates 
folds <- createFolds(dataset$Class, k=5)
str(folds)
Accuracy <- 0

ListFoldsTrain <- list()
ListFoldsTest <- list()

for (i in 1:5){
  
  trainingset<- dataset[-folds[[i]],]
  ListFoldsTrain[[i]]<- trainingset


  testset<- dataset[folds[[i]],]
  ListFoldsTest[[i]] <- testset

#run classification tree model
  tree.1 <-     rpart(Region~palmitic+palmitoleic+stearic+oleic+linoleic+eicosanoic+linolenic+eicosenoic, data=trainingset)


  #Confusion matrix
  tabletree <- table(trainingset$Region, predict(tree.1, type="class"))

  #Accuracy for each fold
  Accuracy[i] = sum(diag(tabletree))/length(trainingset[,1]);Accuracy[i]

}

#Accuracy for each fold
print(Accuracy)

But now how can I get the accuracy of the tree model for the test set?