What is the relationship between 10-fold CV RMSE and test set RMSE score?

Question

In my opinion, these two RMSEs are proportion related. I use the training set to get 10-fold CV RMSE and get a model on the whole training set to predict on the test set. As the training set is larger than the 10-CV training set, the RMSE score on the test set should be better than the 10-CV training set. However, I get a different result: sometimes a bigger improvement model with optimized parameter on 10-CV may perform worse on test set. Why is it so?