# Cross-Validation on a Multinomial Logistic Regression

I am coding in R, and I would like to verify that my multinomial logistic regression model is not overfit and to assess the performance of my model. The data that I am using is categorical and has many different variables (ie, age, cell count), but I selected only the ones I felt would be most relevant to the model.

I know that cross-validation is among the most optimal ways to accomplish this, but I am very new to this process. I poked around online for some examples, but a large portion of the resources I came across only explained cross-validation for models that used two variables. I did, however, learn that k-fold cross is a standard of cross validation--would it be applicable to a multinomial logistic regression model? What other methods are recommended?

Here are the lines of code that I have attempted:

k.folds <- function(k) {
folds <- createFolds(PROJECTONE$categories, k = k, list = TRUE, returnTrain = TRUE) for (i in 1:k) { model <- multinom(categories ~ Age + Cell + Net + Per1 + Per2 + Per3 + Per4 + Lat + ML, data = PROJECTONE[folds[[i]],], method = "class") predictions <- predict(object = model, newdata = PROJECTONE[-folds[[i]],], type = "class") accuracies.dt <- c(accuracies.dt, confusionMatrix(predictions, PROJECTONE[-folds[[i]], ]$categories)\$overall[[1]])
}
accuracies.dt
}
set.seed(500)
accuracies.dt <- c()
accuracies.dt <- k.folds(5)
accuracies.dt

mean.accuracies <- mean(accuracies.dt)


[1] 0.7128463 0.7187894 0.7153652 0.7141058 0.7229219

as my result. I assume this is the confusion matrix that was created. Is there a way that I can better visualize these results, like plotting a graph? If so, what functions/lines of code should I use?

I have also tried repeated k-fold cross validation:

set.seed(500)
v <- c()
v <- replicate(200, k.folds(5))
accuracies.dt <- c()
for (i in 1 : 200) {
accuracies.dt <- c(accuracies.dt, v[,i])
}

mean.accuracies <- mean(accuracies.dt)
lci <- mean(accuracies.dt) - sd(accuracies.dt) * 1.96
uci <- mean(accuracies.dt) + sd(accuracies.dt) * 1.96


Both of these are examples I found off of the internet. However, I received a large response from the repeated k-fold cross validation (as in thousands of results generated). Is that normal? And same question as above--how can I better visualize the results for the model?

Are there any other ways to cross validate this multinomial logistic regression model? After I finish this process, I need to cross validate a boosting, random forest, decision tree, and many other models. Any suggestions on how to approach the rest of these as well?

Thank you for taking the time to help out a newbie. I know I asked a lot of questions, so if anything is unclear please tell me so.