I'm relatively new to machine learning, and most of my experience at this stage comes from working with an automated machine learning tool called DataRobot. In their tool, and in their documentation and tutorials, they promote the idea of using 5-fold cross-validation AND a holdout set; that is, by default about 20% of the data is set aside to be tested against later (it is never used in the training process), and the remaining 80% is split into 5 partitions and run through cross-validation.
I've looked high and low for other references that recommend doing this, and can't find any. Generally, people seem to say that cross-validation is enough, and that there's nothing to be gained from having a holdout set if you're already cross-validating. On the other hand, I could see an argument being made that, because we're fine-tuning hyperparameters, and in DataRobot's case, comparing many different kinds of models, it is possible that the models, as assessed by cross-validation scores, may be overfitting to the data, so it is useful to have a truly independent set (the holdout) to evaluate once model selection is complete.