# R caret classification - why doesn't model accuracy equal accuracy given by predict()?

I have a dataset with 1000 samples, and each sample is 1 of 3 classes. I'm training classifiers on the dataset and predicting classes (5-fold cross-validated) and I'd like to know how well each classifier is doing. To do so, I train the classifiers with caret's train function which returns an accuracy (caret::train(...)$results$Accuracy). I also manually calculate the accuracy using each classifier's predicted classes (stats::predict()).

However, these two ways give different numbers. Why is there a difference? Which method should I use?

Code below reproduces the difference, although the magnitude is really small. In my real dataset the difference is much bigger, e.g. 80% vs 100%.

# make data
df = data.frame(x1 = runif(1000),
x2 = runif(1000),
label = character(1000), stringsAsFactors = F)
df$$label[1:500] = "A" df$$label[501:900] = "B"
df$$label[901:1000] = "C" df$$x1[df$$label=="A"] = df$$x1[df$$label=="A"] - .25 df$$x2[df$$label=="B"] = df$$x2[df$$label=="B"] + .25 df$$x1[df$$label=="C"] = df$$x1[df$$label=="C"] + .125 df$$x2[df$$label=="C"] = df$$x2[df$label=="C"] - .125 # classify ctrl = trainControl(method = "cv", number=5, classProbs = F) mod = caret::train(x=as.matrix(df[,1:2]), y=df$label,
method = "svmLinear",
trControl = ctrl)

# accuracy from mod$$results$$Accuracy
mod.accuracy = max(mod$$results$$Accuracy)

# accuracy from predict()
preds = stats::predict(mod, as.matrix(df[,1:2]), type = "raw")