What misused statistical terms are worth correcting?

Question

Statistics is everywhere; common usage of statistical terms is, however, often unclear.

The terms probability and odds are used interchangeable in lay English despite their well-defined and different mathematical expressions.

Not separating the term likelihood from probability routinely confuses physicians trying to quantify the probability of breast cancer given a positive mammography, “Oh, what nonsense. I can’t do this. You should test my daughter; she is studying medicine.”

Equally spread is the use of correlation instead of association. Or correlation implying causation.

In Al Gore's famous documentary An Inconvenient Truth, a slide illustrates the correlation of ice core $\small \text{CO}_2$ and temperatures, leaving the more technical work to prove causation out of the discussion:

QUESTION: Which statistical terms pose interpretation problems when used without mathematical rigor, and are, therefore, worth correcting?

Answer 1

It can be futile to fight against shifts in language. But

parameter does not mean variable

In classical statistics, which in this case starts precisely with R.A. Fisher who first used the term with this meaning, a parameter is an unknown constant to be estimated, say a population mean or correlation. In mathematics, there are related but not identical meanings, as when a curve is given parametrically. In many sciences, parameter is just another word for a measure (itself a term dense with mathematical meaning), property or variable, say length or conductivity or porosity or virtue, as the case may be. Naturally, an individual's length or virtue is unknown before it is measured. but statistically minded people can be bemused by its use for a set of such measurements. In ordinary or vulgar parlance, parameters (almost always plural) often mean the limits of something, say a personal relationship or a political policy, perhaps stemming from some original confusion with perimeter. With high prior probability it is to be presumed that Bayesians will speak for themselves on their own usages (grateful nod to @conjugateprior).

skewed does not mean biased

For a century or more, skewness has had a specific statistical sense of referring to asymmetry of distributions, whether assessed graphically, measured numerically, or presumed theoretically as a matter of faith or hope. For much longer, or so it may be guessed, bias has meant being wrong on average, which -- so long as we know the truth, meaning a true or correct value -- can be quantified as systematic error. Skewed in ordinary language has a common sense of being warped or distorted, and thus of being incorrect, wrong and so also biased too. That sense (so far as I have noticed, only quite recently) has begun filtering back into statistical discussions, so that the original meaning of skewness is in some danger of being blurred or submerged.

correlation does not mean agreement

Correlation has attracted several precise senses in statistics, which have in common an idea of a bivariate relationship perfect in some precise sense: the leading cases are linear and monotone relationship. It is often diluted, even in statistical discussions, to mean almost any kind of relationship or association. What correlation does not mean, necessarily, is agreement: thus $y = a + bx$ implies Pearson correlation of $1$ or $-1$ so long as $b \ne 0$, but agreement $y = x$ requires the very strict condition $a =0, b= 1$.

unique does not mean distinct

It is quite common to talk about the distinct values of data as unique, but unique is still ideally better preserved as meaning occurring once only. My own guess is that some of the blame stems from the Unix [sic] utility uniq and its imitators, which reduce possibly repeated values to a set in which each value really is unique. The usage, on this guess, conflates input and output of a program. (Conversely, if we talk of duplicates in data, we rarely restrict ourselves to doubletons that occur precisely twice. The term replicates would make more sense linguistically but has been pre-empted for deliberate replication of controls in experiments; the resulting response values are usually not at all identical, which is much of the point.)

samples are rarely repeated

In statistics, a sample includes several values, and repeated sampling is a high theoretical virtue, but one rarely practised, except by simulation, which is our customary term for any kind of faking in silico. In many sciences, a sample is a single object, consisting of a lump, chunk or dollop of water, soil, sediment, rock, blood, tissue, or other substances varying from attractive through benign to disgusting; far from being exceptional, taking many samples may be essential for any serious analysis. Here every field's terminology makes perfect sense to its people, but translation is sometimes needed.

error does not usually mean mistake; as Harold Jeffreys pointed out, the primary sense is erratic, not erroneous.

Nevertheless, we should be wary of our own sins or quirks of terminology:

expected values or expectations (for means over the possible outcomes) may not be what you expect at all, and could even be impossible: in tossing a die fairly with outcomes 1 to 6, the expected value is 3.5

regression is not going backwards

stationary does not mean immobile or fixed

confidence has nothing to do with anyone's mental or psychological state

significance has only sometimes its everyday meaning

exact is often an honorific term, referring to a conveniently tractable solution or calculation rather than one appropriate to the problem

right-skewed distributions to many look skewed left, and vice versa (and the terminology of right and left for skewness assumes that you are looking at something like a conventional histogram, with horizontal magnitude axis)

the lognormal is so called because it's an exponentiated normal

but the lognormal is more normal than the normal

the Gaussian was discovered by De Moivre

Poisson didn't discover the Poisson, let alone Poisson regression

the bootstrap won't help you with your footwear

the jackknife doesn't cut

kurtosis is not a medical condition

stem-and-leaf plots don't refer to plants

a dummy variable is useful, not pointless or stupid

who on Earth (or anywhere else) thinks that heteroscedasticity is really a preferable term over unequal variability?

robust now has at least two major technical meanings for different groups, neither of which inhibits its frequent use, even in technical discussions, to mean merely something like "asserted to behave well"

IV now has at least two major meanings for different groups

factor now has at least two major meanings for different groups

normalize and standardize have uncountably many meanings (we really need to standardize there)

versus describing a graph means vertical variable versus horizontal variable, unless it means the opposite

and (last but not least, to coin a phrase) statistics has at least three major meanings.

Notes:

1. Despite any appearances to the contrary, I think this is a good, serious question.

2. Fashions shift. Well into the twentieth century, it seems that many people (no names, no pack-drill, but Karl Pearson could be mentioned) could only invent terms by reaching for their Greek and Latin dictionaries. (It would be unfair not to give him credit for scatter plot.) But R.A. Fisher did hijack many pre-existing English words, including variance, sufficiency, efficiency and likelihood. More recently, J.W. Tukey was a master in using homely terms, but few should feel distress that sploms and badmandments did not catch on.

3. One comment is based on recollection of "Life is [...] Multiplicative rather than additive: the log normal distribution is more normal than the normal." Anon. 1962. Bloggins's working rules. In Good, I.J. (Ed.) The scientist speculates: an anthology of partly-baked ideas. London: Heinemann, 212-213 (quotation on p.213).

Answer 2

Some of the things I encounter:

1. Treating significance level and CI coverage probabilities as interchangeable, so that people end up doing things like speaking of "95% significance".

   [What's worse is when people who make such errors point to their lecture notes -- or even textbook -- as support for this; in other words the mistake is not theirs, but is being compounded a hundredfold or many-thousands-fold, and worse, even if they understand it correctly, they may actually have to repeat the error anyway, to pass the subject.]

2. There's also a common tendency to think that "significance" somehow exists outside a specific hypothesis/question (leading to questions like "are my data significant" without any clear notion of what question is to be addressed). [A related issue is the "what test should I use for these data?" as if it were the data - rather than the question to be answered - that's the driver of choice of analysis. (While the "design" of the study can impact the specific tests used, the question of interest is more important -- for example, if you have three groups available but your question of interest only relates to a comparison of two of them, the fact that you have three doesn't force you to do a one-way type analysis rather than a straight comparison of the two groups of interest ... as long as your choice of analysis doesn't derive from what the data show. Ideally you plan your questions and analyses before you have data, rather than throwing analysis at data and see what sticks, which it seems post-hoc analysis questions - including "what test should I use for these data?" - tend to lead to.)

3. An occasional tendency to refer to the complement of p-value as some sort of "confidence in", or "probability of" the alternative.

4. "nonparametric data"; another one unfortunately found in a couple of books (and, sadly, in an article that purports to correct a common error) this one comes up so often that it's in my short list of automatically generated comments (which begins "Data are neither parametric nor nonparametric; those are adjectives that apply to models or techniques...") (thanks Nick Cox for reminding me of this particular bugbear)

   Usually what is intended is "non-normal data" but parametric doesn't imply normal, and having approximate normality doesn't imply we need parametric procedures. Similarly, non-normality doesn't imply we need non-parametric procedures. Occasionally, what is intended is "ordinal data" or "nominal data" but in neither case does that imply that finite-parametric models are inappropriate.

5. A common tendency to misunderstand the meaning of "linear" in "linear model" in a way that would be inconsistent with the use of the term "linear" in "generalized linear model". This is partly the fault of the way we use terminology.

6. conflating the mean-minus-median kind of skewness with third moment skewness, and conflating a zero in either (or even both) with symmetry. Both errors are frequently found in basic texts widely used in some particular application areas. [There's a related error of treating zero skewness and zero excess kurtosis as implying normality]

7. this one is so common it's becoming hard to call it an error any more (due in part to the efforts of a particular program) -- calling excess kurtosis simply "kurtosis"; a mistake pretty much guaranteed to lead to communication problems.

Answer 3

"Data" is plural. (The singular is "datum").

Answer 4

Percent vs. Percentage Points: If something increases from 1% to 2%, it increased by 100%. Or: you can say that it increased by 1 percentage point.

Stating that the increase was 1% is very misleading.

Answer 5

While not strictly a statistical term, I vote to retire endogeneity. It's used to refer to everything from reverse causation through confounding to selection and collider bias, when all people really want to do is say: "That effect is not identified".

Answer 6

"Regression towards the mean" does not mean that if we've observed a certain number of iid samples below expected value, the next iid samples are likely to be above the expected value.

Answer 7

Kurtosis does not measure "peakedness."

By definition, it is the expected value (average) of $Z^4$.* Thus, $|Z|$-values less than 1 (corresponding to data values within one standard deviation of the mean, where any "peak" would be) contribute very little to kurtosis; nearly all the contribution to kurtosis is from $|Z|$-values greater than 1 (outliers in particular). See http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4321753/ , Figures 2 and 3 in particular.

* Subtract 3 or not; it makes no difference to this point.

Answer 8

I find abbreviations that aren't clearly indicated are a real problem. For example, I see things like GLM and nowhere is it specified if this means general linear model or generalized linear model. Once can usually figure out what is being referenced after digging into the context, but I find this is particularly troublesome for students just starting to learn about statistical models.

Another example of this is IV. Does this mean instrumental variable or independent variable? Often times it's not made clear until you examine the context.

Something else I see confusion over are "moderator" and "interaction." Also, population (as in the population in general) and the population of interest seems to confuse new students unless it is made very clear.

Answer 9

One that is common in everyday language:

average

To the average person out there (bitter irony fully intended), the mean, median, mode and expected value of anything seem to be the same. They have a natural tendency to do a point estimation, with the unconscious and unassailable assumption that there is an underlying normal distribution. And the equally unconscious assumption of a very small variance. The belief that such an estimation 1) exists and 2) will be very useful for them, because they can take it as a practically certain predictor, is so ingrained, that it is basically impossible to convince them otherwise.

For a real-world example, try to talk to a cook who is asking "what is the average size potato", absolutely certain that if you tell him a number, he will be able to use this one for any recipe which specifies a number of potatoes, and have it come out perfect every time. And getting angry at you for trying to tell him "there is no such number". Sadly, it happens in situations with much higher stakes than making soup.

Answer 10

Linear means:

• Line-like. As in $y = a + bx$ from elementary algebra. In this respect nonlinear means things like $y = a + bx + cx^{2}$, and also things like $y = ax^{b}$

• Linear in the parameters being estimated. As in a regression model (linear, logistic, GLM, etc.) entails a sum of products of scalar parameters and independent variables. In this respect nonlinear means things like $y = \frac{e^{a+bx}}{1+e^{a+bx}}$, and $y = a + bx +x\max (x-\theta,0)$.

• Linear meaning the opposite of dynamic. As in whatever a dependent variable is a function of, it is not a function of its own previous values. In this respect nonlinear means things like $y_{t} = a + by_{t-1} + cx$, and $y_{t}-y_{t-1} = a + b(y_{t-1} - x_{t-x}) + c(x_{t}-x_{t-1}) + dx_{t-1}$.

Where $y$ is a dependent variable, $x$ is an independent variable, and $a, b, c, d$, and $\theta$ are parameters in all the above examples.

Answer 11

Fixed effects and random effects can mean different things for different people. In econometrics fixed effects are actually random and when you think about it every effect in statistics is random so naming something random does not give any meaningful additional information.

Answer 12

The question was about uses of statistical terms that we should CARE to correct. I have been correcting my millennial kids' use of the term 'random' to mean things that are the opposite of random for 10 years now. Considering how many of my trainees struggle to produce a random data sample that is actually random, which happened even before this use of the word, the obfuscation of this term in everyday slang is a crisis.

From the OnlineSlangDictionary:

Definition of random


random

adjective

• unexpected and surprising.

  All of the sudden this guy jumped out from behind the bushes, it was so random!
  The street cleaner never comes down our street. How random.
  

• unexpectedly great.

  The party was totally random.
  

Answer 13

There are already too many great examples mentioned by Glen and Nick... not much left!

Some aspects of regression

• error term and residual (it is somewhat funny when people are proud their residuals are uncorrelated with the regressors)

• prediction and estimation (should we even stop making the distinction when they are about the predicted random effects?)

• prediction/forecast interval versus confidence interval. I think there is a probability > 0.5 to quote the wrong one.

• regressor (column in the design matrix) versus covariable et al. Especially in technical situations where the distinction is essential, many people (including myself) tend to be imprecise.

Answer 14

In insurance environments especially, it is common to use variance to refer to any sort of difference, rather than the mean of the squared differences between each data point and the mean of the data set.

Answer 15

Concerning regression models, many newbies confuse multiple with multivariate regression. It has been stated elsewhere on this site and it may be worth reiterating here as well:

Multiple regression refers to a regression model with more than one covariate (aka independent variable) and a single response (aka dependent) variable.

Multivariate regression refers to a regression model dealing with a vector-valued response.

NaN is different from a missing value, commonly denoted by NA. A NaN is typically delivered by a statistical algorithm that fails to compute a numerical value as expected (e.g. when a division by 0 or the log of a negative number is attempted). An NA happens when, for some reason, the requested information is not available (e.g. in questionnaire data a person refuses to provide his/her income.)

A dataset with NAs is not equivalent to the same dataset with NAs replaced by some arbitrary number such as -999 or -9999.

Answer 16

One of the most harmful misnomers is "type I error rate". $\alpha$ is not a rate (rates can be larger than 1.0) and it is not the probability of being in error. It is best called type I assertion probability to make it clear how limited the concept actual is. More here.

Answer 17

Risk

Risk does not mean probability

Risk is the sum of the costs of all outcomes, each of these costs multiplied by the probability of them happening.

Risk is usually weighed against reward which is the gain that we are seeking to achieve.

Here is one example: How Deadly Is Your Kilowatt. Here the risks — number of dead people for different sources of energy — are weighed against the reward — terawatt hours of energy produced by these sources of energy.

So for instance: the risk of nuclear power is not the probability that a meltdown will happen; it is the probability that a meltdown will happen, multiplied by the number of people that die from it, summed with with the number of people that die from normal operations multiplied by the probability that operations remain normal.

Answer 18

Bayesian

Students learning it might not have trouble telling you whether something "looks" Bayesian, but ask them to solve a problem with a frequentist and a Bayesian approach and they'll probably fail.

In my experience students end up being taught that it's just a philosophical difference, with no concrete example that shows the same problem being attacked with both approaches.

Now ask them why someone might take a frequentist approach in their example; chances are their best explanation would be something like "well, back in the old days, computers didn't exist..."

Answer 19

Statistical terms probably should not be corrected if they are not being used in a specifically statistical context and the terms have colloquial meanings that are consistent with the message communicated.

For instance in the case of the Al Gore plot of temperatures and atmospheric CO2 content, "correlation" can mean simply "a mutual relationship or connection between two or more things.", which is exactly what Gore was suggesting, there are mutual relationships between global temperatures and atmospheric CO2 (the causality is in both directions). So it would be at best pedantry to substitute the statistical definition of correlation where there is no strong reason to think that was what Gore actually intended.

To take it further, in "correlation does not imply causation", "imply" has colloquial meanings which include "indicate the truth or existence of (something) by suggestion rather than explicit reference.", so outside statistics, correlation may well imply causation, depending on what was meant by "imply". Clearly even within statistics, correlation is evidence for causation. Indeed we can only sense correlation (things happening together) not causation (Hume) so correlation (in a colloquial sense) is usually how we arrive at causation.

“Correlation doesn’t imply causation, but it does waggle its eyebrows suggestively and gesture furtively while mouthing ‘look over there.’” Randal Munroe

So if we are going to correct misuse of statistical terms, we need to make sure first that they are being used in a specifically statistical context, and even then we should be trying to understand the substance of the message before worrying about the wording.

Answer 20

Statistical significance & p-values

According to me, the most harmful words in the history of statistics are p-value and statistical significance. They should not only be corrected, but even deleted. The journal of Basic and Applied Social Psychology banned them (Trafimow & Marks, 2015). "Statistical significance" doesn’t mean anything, for any definition that I have heard, even with definitions that didn’t include p-values. The only similar expressions that may be meaningful are "clinically significant", "mechanically significant", "biologically significant", "YOUR_APPLICATION significant", etc.

Statistics are just numbers. A number becomes significant in light of an application IRL, it has no meaning by itself. So statistics, as a field, doesn’t provide significance; it provides numbers. If you obtain such and such statistic, it means that basing a decision on it, following a particular logic, will have a particular impact IRL. The significance of your statistic-based decision, and thus of the statistical value of interest, is the impact of that decision IRL. Thresholds 0.05, 0.01, 0.001, 0.0001, etc. are completely arbitrary and do not come from an impact analysis on real life. Impact analyses start with effect sizes ("The drug has this effect, which reduces symptoms by P%, which would have XYZ impact on patients’ health, etc.") or predictive values, for instance.

A flawed framework

Moreover, p-values and statistical significance are based upon a flawed framework. The null hypothesis significance testing (NHST) procedure has been thoroughly criticized through decades (see The ASA's Statement on p-Values: Context, Process, and Purpose and references). It’s not about Bayesians vs. frequentists, it’s just flawed. No Bayesian alternatives that I read convinced me, despite the fact that attaching probabilities to hypotheses can be meaningful in some contexts (e.g., when supposing that the drug that you’re studying is ineffective on some patients, or has a different effect).

Fixing things

Descriptive statistics and performance metrics are enough to evaluate the quality of a model/hypothesis. If you want to test the generalizability of the model/hypothesis, you need to repeat the experiment on a new population of interest. Studies after studies, experiments after experiments, you’ll likely find better estimates of the reality. This is systematic empiricism, or science in other words.

For example, systematic empiricism is how our predecessors improved their understanding of gravity. The first humans proposed the simple model "object falls to the ground". Then, they started building catapults and formalized more and more the process of falling, taking into account the mass of the object, the distance, the time it would take to fall, etc. Centuries after, Newton did precise tests using more elaborate mathematical tools and created a model including Newton’s laws of motion. But Einstein wasn’t satisfied with it, so he proposed the theory of general relativity, etc.

These theories, which have been being more and more precise, have never needed a p-value or "statistical significance" to be validated. You simply need to measure where objects fall, admit it’s the truth, and then build a model that predicts it. The challenge is to build a model that predicts well in different context. In any case, any fall you observe remains the truth. If you change the angle of your throw, it will fall somewhere else. You don’t need to run a "significance test" on a sample of falls to know whether the objects have fallen somewhere else, you’ve just observed where they have fallen. The only statistics you need are those who describe the falls (where, when, with what angle, etc.) and the difference between your prediction and the reality, whereupon you build a model that can reduce this difference. P-values and "significance tests" don’t tell you where object falls (what is the reality), they don’t tell you if your predictions were right, and they don’t tell you how to reduce the difference between the prediction and the reality. So ban them.