I can see lots of paper mentioning they selected some parameters like regularization parameter $\lambda$ by cross validation. What do they mean by that?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Optimal values for things like regularization parameters cannot (always) be computed in advance. Often, we are forced to pick a value and see what happens (e.g. train a model with a given value and assess its performance).
Cross-validation is a very popular method to test the performance for a given value of the parameter $\lambda$. The value which yields the best cross-validation performance is then selected.
answered May 20, 2013 at 20:04