1
$\begingroup$

Are there any reference document(s) that give a comprehensive list of misnomers in machine learning? I would like to have a list and simple explanation if needs be that I could go through easily (vs. a full encyclopedia) to make sure I know all the jargon pitfalls.

Examples:

  • logistic regression: actually not regression, but classification (using the ML terminology).
  • multilayer perceptron: the model actually comprises multiple layers of logistic regression models (with continuous nonlinearities) rather than multiple perceptrons (with discontinuous nonlinearities). (see Christopher M. Bishop's Pattern Recognition and Machine Learning, page 226).
  • non-parametric model: there are actually parameters, but potentially an infinite number of them. The difference between parametric model and non-parametric model is that the former has a fixed number of parameters, while the latter grows the number of parameters with the amount of training data. The non-parametric model is not none-parametric: parameters are determined by the training data, not the model (see Wikipedia). E.g. Bayesian nonparametric models are not parameter free, but have an infinite number of parameters.

I'm aware that there is some dose of subjectivity when declaring a term as a misnomer. I prefer to have higher recall and lower precision, to a reasonable extent.

$\endgroup$
  • 2
    $\begingroup$ Why is logit not a regression? Some of your examples are odd. $\endgroup$ – Aksakal Aug 25 '15 at 19:01
  • 1
    $\begingroup$ @Aksakal: Andrew Ng's lectures make this note, which was puzzling to me as well. I think he means that in the ML community, the term regression refers to continuous outcomes, while classification refers to categorical outcomes. Of course, this terminology is not consistent with statistical terminology. $\endgroup$ – Cliff AB Aug 25 '15 at 19:05
  • $\begingroup$ @Aksakal Same as Cliff AB. What's the issue with the two other examples? $\endgroup$ – Franck Dernoncourt Aug 25 '15 at 19:06
  • 1
    $\begingroup$ @FranckDernoncourt, your statements about nonparametric models are unsual, if not outright wrong $\endgroup$ – Aksakal Aug 25 '15 at 19:08
  • 4
    $\begingroup$ A machine learning person is someone who uses ML to mean machine learning. A classical statistical person is someone who uses ML to mean maximum likelihood. (Part serious, part facetious.) $\endgroup$ – Nick Cox Aug 25 '15 at 19:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.