# Most confusing statistical terms

We statisticians use many words in ways that are slightly different from the way everyone else uses them. This cause lots of problems when we teach or explain what we are doing. I'll start a list (and now I'll add some definitions, per comments):

• Power is the probability of correctly reject a false null hypothesis in a hypothetical situation where the data comes from a specific alternate hypothesis or range of alternates. Usually, this means "our statistical method should succeed" if "something is happening".
• Bias - a statistic is biased if it is systematically different from the population parameter associated with it.
• Significance - results are statistically significant at some percent (often 5%) in the following situation: If the population which the sample comes from has a true effect of 0, a statistic at least as extreme as the one gotten from the sample would only occur 5% of the time.
• Interaction - Two independent variables interact if the relationship between the dependent variable and one independent variable is different at different levels of the other independent variable

But there have to be many others!

• I would suggest people add some greater context to their answers as well. Examples might be the same word used differently in different fields (fixed effects Gelman, 2005) or words that have different meanings in different contexts (significance vs. statistical significance Bushway et al., 2006). Jan 12 '12 at 13:02
• It would be good if answerers could explain in one or two sentences what the "technical" term really means, or what might cause it to be perceived as having a different meaning.
– chl
Jan 12 '12 at 13:02
• I will complete my answer according to your comments later on ;-) Jan 12 '12 at 13:20
• ... and "correlation" ! Jan 12 '12 at 15:59
• For "sample," see the comments to stats.stackexchange.com/questions/20945/….
– whuber
Jan 12 '12 at 16:49

"Significant" is the biggest one I run into, because it has both a common English-use meaning and that meaning will crop up in the discussion of research results. I even find myself mixing in "significant" to mean important in the same sentence where I've talked about statistical results.

• Right, but there's no better word for "I'm pretty damn sure it's significant, but I haven't done any tests on it, and won't, because it's obvious/can't be done/whatever" Apr 17 '12 at 5:29
• The question to keep asking -- myself, yourself and others -- is But what does it signify? Mar 22 at 9:24

I would suggest adding Linear to the list.

I asked a question on math.SE about what I, as an engineer, think of as linear minimum mean square error estimation of a random variable $Y$ given the value of a random variable $X$ (meaning estimating $Y$ as $\hat{Y} = aX+b$ with $a$ and $b$ being chosen so as to minimize $E[(Y-aX-b)^2]$), and gave a partial answer. One of the comments on the question said

"I am somewhat uncomfortable with your language, since I fear that this way of using the word "linear" might feed into the popular misunderstanding that the reason why linear regression in called linear regression is that one is fitting a line. People who think that then find it confusing when a statistician insists that one is doing linear regression when one fits a parabola or a sine wave, etc."

So, what does linear regression mean to a statistician?

• Related question on the site in reference to this answer, What does linear stand for in linear regression? Jan 12 '12 at 14:01
• @AndyW So would you say that Linear belongs on the list that Peter Flom started, or not? Jan 12 '12 at 14:19
• yes I agree it fits the bill for this list. (+1) Jan 12 '12 at 14:48
• It fits the list, but for an unusual reason: the meaning of the term "linear" is well established and used consistently across many mathematically-oriented fields. The potential confusion concerns which part of the formula is linear.
– whuber
Jan 12 '12 at 21:16
• Can you give an example of how you fit a parabola and still call it a linear model? Mar 27 '18 at 16:08

"Confidence"

It's very hard to dissuade non-statisticians that their confidence interval is not (directly) a statement about the credibility of different parameter values.

To have confidence, in the technical meaning of the term, we need to imagine some set of repeated experiments, each one computing an interval in some pre-specified way. To be a 95% confidence interval, 95% of these uses of the formula will trap the relevant parameter of interest.

But non-statisticians routinely interpret "95% confidence" to be a statement about plausible parameter values, based on one experiment alone. Typically, they assume that the interval covers 95% of some posterior beliefs about the parameter, i.e. "we're pretty certain the parameter is between $a$ and $b$". This instead defines a credible interval.

(There are of course situations when the two notions agree, either approximately or exactly. But in general they don't, and numerical agreement doesn't remove the problem of misuse of technical terms.)

probability

It seems to me that most of the problems associated with interpreting hypothesis tests and confidence intervals stem from the application of a Bayesian definition of "probability" when the procedure is based on a frequentist one. For example the p-value being the probability the null hypothesis is true, when AFAICS no probability can be associated with the truth of a particular hypothesis in a frequentist setting.

• It looks like the same consideration would apply for those who are used to say that the (true) parameter has a 95% chance of lying between xx and xx, when speaking of/interpreting confidence intervals.
– chl
Jan 12 '12 at 15:45
• yes absolutely! Jan 12 '12 at 15:51
• +1 I would phrase your last sentence slightly differently, however. Within the frequentest setting the probability that the null hypothesis is true is either 1 or 0, but you don't know which. (Strictly speaking, this is not quite right, because 'probability' is a long-run relative frequency and 'long-run frequency' doesn't really apply. Nonetheless, when phrased this way people can understand what's being said / how we understand the situation we're in. For example, people realize that you can't use the null hypothesis p-value as the probability that the null hypothesis is true.) Jan 12 '12 at 17:18
• "because 'probability' is a long-run relative frequency" Lots of probabilists will vehemently dispute that statement Jan 12 '12 at 18:31

"Likelihood" -- it is synonymous with "probability" in everyday speech, but in Statistics it has a special meaning: it is a function of the parameters of a statistical model and a particular data situation, whose value is the probability of the observed outcome assuming that the parameters are equal to the parameter values.

• In turn that's a loose statement when applied to continuous variables, as the likelihood then depends on probability density. Yet further, constant factors that shift the likelihood up or down but don't affect the position of the maximum are often omitted. Mar 22 at 9:27

Error.

In statistics, an "error" is a deviation of an actual data value from the prediction of a model.

In real life, an error is a spllng mstake or other goof.

• Isn't a spelling mistake just a deviation from the actual (intended) value of the communication medium? I don't really see how this is a different word, it's just that it's used in a different (but not conflicting) context. I find it hard to believe that it would be all that confusing to someone new to the field. Apr 17 '12 at 5:21
• One reason why a value can differ from a prediction is that the experimenter messed up. That is like a spelling mistake. But why is your weight different than the average weight of all people of your gender and age? Why is your income different than the mean income? In statistics, this deviation from a mean is an "error" but it isn't a mistake, just variation. Apr 17 '12 at 18:50
• True, but it depends how you look at it, I think. if you look at the spelling of a word over a sample of a primary school, you're going to get variation, caused by humans, yes, but nor caused by the experimenter. You could look at the same thing over writing in english from different ages. I reckon you'd find that early english had a lot higher variability :) Apr 18 '12 at 0:11
• @HarveyMotulsky: analytical chemistry uses error in both ways. We speak of systematic error, random error and gross errors. Textbook: "gross errors can be avoided". May 6 '19 at 16:15
• Harold Jeffreys pointed out that the primary statistical sense is erratic, not erroneous. Otherwise put, error is uncontrolled variation and we are usually ignorant about any true value existing independently of the data. Mar 22 at 9:30

"Inference"

One of the hardest things for me to understand at first was the difference between a population and a sample. Statisticians write these fancy population level regression equations and then all of a sudden drop down into sample level work and the $\beta$s become $b$s. It took me a long time to realize that you were using the sample level data and regression equations to estimate the population level parameters.

Another important part about inference is the central limit theorem. Once you realize that you are simply sampling from a population -- although sampling is another complicated feature akin to inference -- then you understand that even if the sample mean holds one value, that value isn't necessarily the same mean as in the population.

Perhaps I took a relatively loose understanding of your question, but once someone understands inference or the differences between a sample and the population then the entirety of statistics opens to them.

To us (or at least me), "randomness" of a "sample" suggests that it is representative of the "population".

To others, "randomness" sometimes implies that a person/thing is unusual.

• I haven't run into that confusion over "randomness". But if you have, then clearly it exists. Jan 19 '12 at 10:38
• More precisely, that it has existed Jan 19 '12 at 13:28
• The latter usage of "random" seems pretty recent to me. I find it a bit annoying for that reason (just makes stats harder for people to understand). It's even more annoying when I hear myself using it in that sense.. Apr 17 '12 at 5:23

I think one should distinguish between terms confusing the public and terms confusing statisticians. The above suggestions are mostly terms well understood by statisticians and (possibly) misunderstood by the public. I wish to add to the list some terms misunderstood by statisticians:

• Bayesian: Originally referred to what is now known as subjective Bayes (a.k.a. epistemic, De Finetti). Today the term will be used anytime Bayes' rule shows up, rarely in the context of subjective beliefs, which is considered decision theory.
• Empirical Bayes: Originally referring to a frequentist setup with a non parametric prior. Today, will typically mean the parameters of the parametric (objective) prior are estimated and not known a priori. I.e., what was once known as type-II maximum likelihood.
• Non parametric: Sometimes refers to "model free". Sometimes to "distribution free". Has become practically uninformative in the days "parametric" models might include millions of parameters.
• Type III error: sometimes referring to a sign error. Sometime referring to a mis-specification of the model.
• When I asked, I intended "terms confusing to the general public" but certainly terms confusing to statisticians are also worth listing Jan 19 '12 at 10:36
• This probably should be split up into separate answers. Apr 17 '12 at 4:08

Ecological, commonly used to refer to biological systems, but also a statistical fallacy. From Wikipedia:

An ecological fallacy (or ecological inference fallacy) is an error in the interpretation of statistical data in an ecological study, whereby inferences about the nature of specific individuals are based solely upon aggregate statistics collected for the group to which those individuals belong. This fallacy assumes that individual members of a group have the average characteristics of the group at large.

• The main use I have seen for this is in discussions of social data, focusing on the difference between individual data and areal data, and not in ecology! Mar 22 at 9:34

Is a "survey" a type of math ("survey sampling") or a piece of paper ("questionnaire")?

I haven't conducted a survey on this, but I suspect that much of the public considers a "survey" to be the latter. I suspect further that they don't think about the former.

• Aren't surveys done by surveyors? ;) Jan 18 '12 at 5:42

I usually find people being quite ad hoc when using them, employing them interchangeably without first explicitly defining what they mean and I have come across papers that refer to "loading vectors" and sometimes mean the PCs themselves and other times the "weights" associated with a specific PC.

Probably the fact that Jolliffe's excellent reference on Principal Components states at the end of section 1.1 "Some authors distinguish between the terms ‘loadings’ and ‘coefficients,’ depending on the normalization constraint used, but they will be used interchangeably in this book." just made people think they have a free pass to mix and match terminology to their liking....

Additive model. Still not really sure what this means. I think it refers to a model without interaction terms. But then I will come across an article where they're using it to refer to something else, i.e. a spline model.

Consistency

First, many other people read into this a notion of something like "does not have any (internal) contradictions", which is related, but surely not equivalent, to definitions used in statistics.

Second, even within statistics, it has more than one meaning, such as consistency of an estimator, consistency of a test or consistent model selection criteria.

One of the terms that I find most confusing is the "confusion matrix". Of course, The term used itself is confusing, not the concept.

I tried to track the history of the term and it is quite interesting too. The confusion matrix was invented at 1904 by (http://en.wikipedia.org/wiki/Karl_Pearson). He used the term http://en.wikipedia.org/wiki/Contingency_table. It appeared at Karl Pearson, F.R.S. (1904). Mathematical contributions to the theory of evolution (PDF). Dulau and Co. http://ia600408.us.archive.org/18/items/cu31924003064833/cu31924003064833.pdf

During War World 2, https://en.wikipedia.org/wiki/Detection_theory was developed as investigation of the relations between stimulus and responds. The confusion matrix was used there.

Due to detection theory, the term was used a psychology. From there the term reached machine learning.

It seems that though the concept was invented in statistics, a field very related to machine learning, it reached machine learning after a detour in during a period of 100 years.

For some references of the usage of the term see: What is the origin of the term confusion matrix?

"Statistics"

To the general public, a substitute for, "now I'm about to lie to you and speak in a way you don't understand."