I'm using a Grid Search CV to find optimal hyperparameters for an SVM. I want to use the best combination of hyperparameters and evaluate model performance using those hyperparameters. Can I just use the average accuracy given by the Grid Search CV as a metric for model performance?
I've looked at SE Post as well as SKLearn link, which indicates that the performance may be biased. However, why would the performance be biased? Isn't the performance metric given by the cross-validation for unseen data (namely, the held-out set in the cross-validation)? And as this unseen data is different for each fold, as well as the trained model, wouldn't the metric given back by the cross-validation be a good representation of how it would perform on unseen data?