2
$\begingroup$

I am looking for a term which denotes the violation of Occam's razor. In other word a term which denotes the specification of an overly complicated model when a simpler model works just as well, or almost as well.

This practice is so common in science, that I feel there must be a term for it. But I have not been able to find this term so far.

In a previous version of this question, I asked if overfitting is this term. But overfitting is overly restrictive, is mainly applicable to predictive modeling, and is not applicable to descriptive or explanatory modeling without explicit cross-validation (see https://arxiv.org/pdf/1101.0891.pdf for a discussion of the differences between descriptive, explanatory and predictive models).

$\endgroup$
  • 1
    $\begingroup$ The idea of overfitting only applies to a sample. If you want to stipulate that your fit has certain properties (ie, "arbitrarily small ϵ") for "all past and future data", then you are referring to the population & overfitting cannot apply. As @MartijnWeterings notes, the quadratic model that fits arbitrarily closely to the true linear population function would have a coefficient of $0$ on the $X^2$ term. $\endgroup$ – gung Dec 31 '17 at 21:21
  • 1
    $\begingroup$ @gung michael-chernick whuber MartijnWeterings thank you for your previous feedback on my question. I have radically rephrased the question. Would be grateful if the hold on the question can be removed. $\endgroup$ – Akim Jan 3 '18 at 16:14
  • $\begingroup$ I think the term overfitting applies to the descriptive practice of a model as well as the predictive. In both cases the intent is to be able to say something about the population (i.e. the process that generated the data). In predictive overfitting, we learn noise as signal, which makes our predictions based on fictitious features of a population, in explanatory overfitting we do the same, which makes qualitative features of our model (i.e. parameter estimates) a function of noise in the data. In both cases, the cause is the same, overfitting. $\endgroup$ – Matthew Drury Jan 3 '18 at 16:22
  • $\begingroup$ @Akim I've nominated your question for re-opening. Thank you for your clarifications. $\endgroup$ – Matthew Drury Jan 3 '18 at 16:23
  • $\begingroup$ @MatthewDrury thank you for the clarification and for nominating. $\endgroup$ – Akim Jan 3 '18 at 16:31
1
$\begingroup$

Model overspecification.

[P]rediction equations [...] may suffer from overspecification. Too many predictor variables in the model can create problems when the goal of the investigation is to make inferences on the model parameters. This is because there is a great danger of redundancy among the predictor variables. If several predictor variables repeat information, it is difficult to determine which of the redundant variables should be retained in the model and which should be eliminated.

Regression Analysis and its Application: A Data-Oriented Approach. Richard F. Gunst, Robert L. Mason. CRC Press 1980. Page 110.

google link

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.