1
$\begingroup$

A terminology problem. In machine learning we have the following problem:

Choosing the optimal model (or training): $$ f^* = \arg\min_{f \in \mathcal{F}} \sum_i l(x_i,y_i) $$

Is the term "model selection" always "exactly" referring to this? Or something else?

$\endgroup$
  • $\begingroup$ No, because model selection can have additional cost constraints (such as reducing complexity of $f$). $\endgroup$ – pat Oct 28 '13 at 17:15
  • $\begingroup$ Could you elaborate? Or could you give me some pointers? $\endgroup$ – Daniel Oct 28 '13 at 17:18
  • $\begingroup$ No. In many approaches, for example SVM, training means solving a given optimization problem which itself is parametrized. Model selection means finding the right parameters. $\endgroup$ – Marc Claesen Oct 28 '13 at 18:05
1
$\begingroup$

The best model is not necessarily the model which minimizes error, but typically attempts to reduce overfitting bias by adding penalties for cost complexity and by cross-validating between training and validation samples.

web.engr.oregonstate.edu/cs534 slides

$\endgroup$
1
$\begingroup$

Training often involves model selection (choice of model structure, set of input variables, transformations, ...). But, as @MarcClaesen pointed out, training also includes the process of fitting the model, i.e. finding best values for its parameters.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.