I thought cross validation error would tend to be higher than the test error for the model fit on the entire data set due to the following reasons:
Cross validation uses only a subset of data compared to the model fit on the entire data set, which contributes to bias.
Cross validation averages the outputs of k fitted models in which k is the number of 'folds' in the cross validation, and there's obviously going to be overlaps between training sets, which contribute to the variance.
My guess was that CV error must always be smaller than the test error for the model fit on the entire data set. However, I heard that it is possible for the CV error to be much lower than the test error. Is there some possible reason?
Also, what could be some possible reasons behind CV error to be much lower than the actual theoretical error?