I have a question regarding parameter tuning in nested cross validation. I have already found some excellent questions and comments on the topic (Training with the full dataset after cross-validation?). However, they do not really address my question.
Suppose I want to do model checking for a SVM classification with a k-fold outer cross validation. For each fold of the outer cross validation I use a nested cross validation for parameter tuning (cost, gamma) of the respective SVM using grid-search. My question is how and where to define the parameter ranges for the grid-search then.
The options I see:
Option 1: Define a range once and use this same range to tune the method in each fold. If this is a valid approach, how would you set the parameter values ex-ante if you have no idea how they perform?
Option 2: Define an individual range in each fold and refine the search manually several times. Also, what would be a good criterion to stop refining the grid-search in each fold? This would also imply that you would need to do the tuning manually in each fold.
Any thoughts on what is a good approach here?