In my statistical teaching, I encounter some stubborn ideas/principles relating to statistics that have become popularised, yet seem to me to be misleading, or in some cases utterly without merit. I would like to solicit the views of others on this forum to see what are the worst (commonly adopted) ideas/principles in statistical analysis/inference. I am mostly interested in ideas that are not just novice errors; i.e., ideas that are accepted and practiced by some actual statisticians/data analysts. To allow efficient voting on these, please give only one bad principle per answer, but feel free to give multiple answers.
$\begingroup$
$\endgroup$
6
-
6$\begingroup$ I don't know whether I would call these ideas "silly". If they are commonly accepted, then their wrongness is obviously not obvious. How about just calling them "worst ideas", or "worst practices"? $\endgroup$– Stephan KolassaCommented Jul 10, 2020 at 10:54
-
1$\begingroup$ Related: stats.stackexchange.com/questions/219471/… $\endgroup$– kjetil b halvorsen ♦Commented Jul 10, 2020 at 13:50
-
1$\begingroup$ Related: stats.stackexchange.com/questions/4551/… $\endgroup$– kjetil b halvorsen ♦Commented Jul 10, 2020 at 14:37
-
5$\begingroup$ ARIMA : A marvel of theoretical rigor and mathematical elegance that is almost useless for any realistic business time series (auto.arima and similar automated tools always p,d,q orders that were just as well handled by a more intuitive ES approach). $\endgroup$– Skander H.Commented Jul 11, 2020 at 0:37
-
5$\begingroup$ @SkanderH.: do you want to post your comment as an answer? Maybe add a pointer to how incredulous the statistics community was when it turned out that ARIMA was no good at forecasting in the first M competition, as summarized very nicely and entertainingly by Hyndman (2020, IJF). $\endgroup$– Stephan KolassaCommented Jul 17, 2020 at 2:36
|
Show 1 more comment
33 Answers
$\begingroup$
$\endgroup$
3
I feel like very basic statistics is not as intuitive as basic school physics. Or maybe there is a much more intuitive base, but I haven't come to know it.
Statistics tells us if there is a correlation, not the mechanisms of causation. And correlation does not imply causation. To tell "why" two phenomena are connected, one has no other way than to find the causation. However, sometimes in classroom lectures or seminars etc., the importance of statistics is described in such a way that leaves a false impression that statistics is used to determine causal relation.
Misuse of statistics happen (intentionally or unintentionally) because statistics gives the place to do so. When things are lengthy, complicated; and the intuitive basis behind those rigorous treatments are not understood; there might be scopes for conceptual fallacies or conceptual mistakes.
A little learning is dangerous, especially when it comes to statistical terms; because statistical terms and graphs could be deliberately misused to establish a "damn lie" into apparent truth.
-
5$\begingroup$ #4 is nonsense often repeated by individuals who conflate the human ability to deceive with some special property of the language of statistics. You can lie with statistics, just as you can lie with any language. $\endgroup$– AlexisCommented Jul 15, 2020 at 19:30
-
$\begingroup$ The first part of #4 (that a little learning is dangerous) is true. $\endgroup$– BenCommented Jul 27, 2020 at 11:20
-
$\begingroup$ From the OP: To allow efficient voting on these, please give only one bad principle per answer, but feel free to give multiple answers. Also, hardly any of the points actually is a bad statistical idea/practice. $\endgroup$ Commented Jul 27, 2020 at 14:15
$\begingroup$
$\endgroup$
Not including ordinal regression within a model when some of the variables are purely ordered categories. I see this a lot with the analysis of Likert-type data. Imposing additive properties and scaling properties that the data does not demonstrably have does not reflect best practice.
In order to be proactive I would like to offer an entry point for people to learn more about ordinal regression. If you're reading this, you are not familiar with ordinal regression, you'd like to learn ordinal regression, and you don't mind a Bayesian approach, then I recommend you watch Statistical Rethinking 2022 Lecture 11 - Ordered Categories.
Caveat: While this topic can be a little nuanced when considering general function approximation (see here), the simpler white-box models that scientists are often building doesn't match with that context IMO.
$\begingroup$
$\endgroup$
1
Doing Proportional Odds Logistic Regression.
Ordinary logistic regression produces correct class probabilities only if the two classes are normally distributed, with a same variance. It is already uncommon enough to check for that.
However, in proportional odds, this condition cannot be satisfied, ever. If the classes $A$ and $BC$ happen to have equal variances, then classes $AB$ and $C$ cannot have them (except in a pathological case where they fully overlap).
-
5$\begingroup$ You are confusing the PO model with something else. The PO model has nothing to do with any of that and definitely does not assume normality. $\endgroup$ Commented Jul 23, 2020 at 18:53