What is “Cross-Validation Error” in plain English?

Question

Say you use Cross-Validation to fit a regression model to a dataset. And you get a bunch of CV-scores/CV-errors (one CV-score/CV-error per choice of ${\it number\ of\ parameters}$ to include in the model). What exactly is a Cross-Validation Error?

Answer 1

In plain English, cross validated error is your best guess for the average error you would see with your regression model on new data.

I'm being a bit fast and loose. Plain English definitions sometimes sacrifice precision for utility. There are lots of "average errors" for which this could refer to: average error keeping the training the same? Average error over new training data? Average error considering the entire hyperparameter selection to cross validation? These are all legitimate and yet different interpretations of "average error". For the time being, I think the definition I provide is good enough for practical purposes.