I just started to learn data mining. I have learned that cross validation can be used to compare models (e.g. svm vs logistic, etc) and to do tuning (like regularization parameter) .
I have a dataset. I split it into training and testing set and then used training to do cross validation to compare different models and then tuning the "winner model" on the same cross validation to get the most out of it.
Will training and tuning on the same data lead to overfit?
(In coursera machine learning forum, I have seen an instructor say that each set of data can only be used for one purpose. If you use cv to adjust the regularization parameter then you cannot use cv to select the polynomial degree.)
I have seen lots of articles saying that cv can be used to compare models and do tuning, but I do not know. If I want to do both, what procedures should I follow? If I have several models in my mind and I use cv to compare those models, then I find out that svm is good, but how should I tune it further? Use the same cv set? or use another validation set to estimate?