I am new to machine learning or R and tried to code a function "smotevalue" in R in order to fine-tune the parameters of SMOTE for binary classification/prediction in imbalanced data. The idea is to vary the two parameters of SMOTE (K and dup_size) in order to optimize the AUC score of predictions for testdata. Something seems to be wrong here, as the optimization function reports perfect AUC scores at every iteration. I tested the code within the "smotevalue" function in isolation, and it also gives me unreasonably high AUC scores when predicting the test data. (For example I changed the data-split to 5%/95% training/test, I still got an AUC of about 0.99 for the predictions on test) Is the testdata somehow used to train the model here? Where is my mistake? # Libraries library(smotefamily) #SMOTE library(caret) #Data Splitting library(pROC) #AUC Metric library(DEoptim) #Differential Evolution Optimization # Set Seed set.seed(123) # Read Training Data Into R bankruptcy.train <- read.csv(file=".../bankruptcy_Train.csv", header=TRUE, sep=",") # Read Test Data Into R bankruptcy.test <- read.csv(file=".../bankruptcy_Test_X.csv", header=TRUE, sep=",") ## SMOTUNE ## smotevalue <- function(x) { inTraining <- createDataPartition(bankruptcy.train$class, p = .05, list = FALSE) training <- bankruptcy.train[ inTraining,] testing <- bankruptcy.train[-inTraining,] data_train <- SMOTE(training[,-65], training$class, K = x[1], dup_size = x[2]) data_train <- data_train[["data"]] data_train <- data.matrix(data_train) data_train <- xgb.DMatrix(data = data_train, label=data_train[,65]) bst <- xgboost(data=data_train, booster = "gblinear", nthread = 4, lambda = 1e-04, alpha = 0, eta = 0.3, nrounds=150, eval.metric = "auc", objective = "binary:logistic") data_test <- data.matrix(testing) data_test <- xgb.DMatrix(data = data_test, label=data_test[,65]) test.pred.bst <- predict(bst, newdata=data_test) roc_obj = roc(testing[,65], test.pred.bst) auc(roc_obj)*(-1) } # Specify That SMOTE Parameters Will Be Tuned In Discrete Steps Integer <- function(x){ x[1:2] <- round(x[1:2]) #k and dup_size -> integer values } # Differential Evolution Optimization of function smotevalue, # varying K and dup_size for highest AUROC value smote_de_obj <- DEoptim(smotevalue, lower = c(1, 1), upper = c(15, 50), control = DEoptim.control(NP = 20, itermax = 50, CR = 0.3, F = 0.7), fnMap = Integer) # Report Tuned SMOTE Parameters fitted_params <- smote_de_obj$optim$bestmem