I have a question regarding the Cross-validation process. I am in the middle of a course of the Machine Learning on the Cursera. One of the topic is about the Cross-validation. I found it slightly difficult to follow. I do know why we need CV because we want our models to work well on future (unknown) data and CV prevents from overfitting. However, the process itself is confusing.
What I have understood is that I split data into 3 subsets: training, validation, and test. Train and Validation is to find optimum complexity of a model. What I don't understand is the third subset. I understand I take a number of features for the model, train it and validate it on Validation subset and look for the minimum Cost Function when I change the structure. When I found it, I do test the model on Test subset. If I have already found minimum Cost Function on Validation subset, why would I need to test it again on Test subset ???
Could someone please clarify this for me?
Thank you