# Grid tuning CV: Full blown vs limited

I am doing grid based parameter search using CV for my XGBoost model in R. Code is for reference but question below is lot generic:

opt1 <- as.integer(3/(sum(base_data$NON_CLR_SLS_Q > 0)/nrow(base_data))) opt2 <- as.integer(1/sqrt(sum(base_data$NON_CLR_SLS_Q > 0)/nrow(base_data)))
searchGridSubCol <- as.data.table(expand.grid(subsample = c(0.75, 1),
colsample_bytree = c(0.6, 0.8),
min_child_weight = c(opt2)
))

cross_val <- function(parameterList){

#Extract Parameters to test
currentSubsampleRate <- parameterList$subsample currentColsampleRate <- parameterList$colsample_bytree
current_min_child_weight <- parameterList$min_child_weight param1 <- list("objective" = "reg:linear", "eval_metric" = "rmse", "max_depth" = 10, "eta" = 0.03, "gamma" = 0, "subsample" = currentSubsampleRate, "colsample_bytree" = currentColsampleRate, "min_child_weight" = current_min_child_weight, "alpha" = 0.1) xgboostModelCV <- xgb.cv(data=train_matrix, label=output_train, nrounds = 500, nfold = 5, verbose = F, early.stop.round=5, prediction=T, param=param1, missing = NaN) test123 <- data.table(rowSubSample=currentSubsampleRate,colSubSample=currentColsampleRate, min_child_weight=current_min_child_weight, numIter=which.min(xgboostModelCV$dt[, test.rmse.mean]),
RMSE=xgboostModelCV$dt[which.min(xgboostModelCV$dt[, test.rmse.mean]),test.rmse.mean])

rm(xgboostModelCV)

return(test123)

}

rmseErrorsHyperparameters <- searchGridSubCol[,cross_val(.SD),by=1:nrow(searchGridSubCol)]

rmseErrorsHyperparameters <- rmseErrorsHyperparameters[order(rmseErrorsHyperparameters\$RMSE),]



I ran my model for nrounds = 500 but time taken when using grid combinations in (unexpectedly) non-linearly growing. So I want to cut time for grid search.

I found 500 is good limit for grid search in my case, as most cases model overfitting start before 500 iterations. So this is full blown CV. However to save time I want to just run each combination for say 100 or 200 iterations and compare (this I am calling limited CV). What is shortcoming of limited CV Vs full blown CV for grid search; apart from not be able to get statistically sound estimate for RMSE (in my case). Can optimal parameters of limited CV can be different than full blown CV if yes then why? In other words is it always necessary to run till overfitting is detected when doing hyperparameter tuning?