# Most confusing statistical terms

We statisticians use many words in ways that are slightly different from the way everyone else uses them. This cause lots of problems when we teach or explain what we are doing. I'll start a list (and now I'll add some definitions, per comments):

• Power is the ability to correctly reject a false null hypothesis. Usually, this means correctly saying "something is happening".
• Bias - a statistic is biased if it is systematically different from the population parameter associated with it.
• Significance - results are statistically significant at some percent (often 5%) in the following situation: If the population which the sample comes from has a true effect of 0, a statistic at least as extreme as the one gotten from the sample would only occur 5% of the time.
• Interaction - Two independent variables interact if the relationship between the dependent variable and one independent variable is different at different levels of the other independent variable

But there have to be many others!

-
I would suggest people add some greater context to their answers as well. Examples might be the same word used differently in different fields (fixed effects Gelman, 2005) or words that have different meanings in different contexts (significance vs. statistical significance Bushway et al., 2006). –  Andy W Jan 12 '12 at 13:02
It would be good if answerers could explain in one or two sentences what the "technical" term really means, or what might cause it to be perceived as having a different meaning. –  chl Jan 12 '12 at 13:02
I will complete my answer according to your comments later on ;-) –  ocram Jan 12 '12 at 13:20
... and "correlation" ! –  Stéphane Laurent Jan 12 '12 at 15:59
For "sample," see the comments to stats.stackexchange.com/questions/20945/…. –  whuber Jan 12 '12 at 16:49

"Significant" is the biggest one I run into, because it has both a common English-use meaning and that meaning will crop up in the discussion of research results. I even find myself mixing in "significant" to mean important in the same sentence where I've talked about statistical results.

-
Right, but there's no better word for "I'm pretty damn sure it's significant, but I haven't done any tests on it, and won't, because it's obvious/can't be done/whatever" –  naught101 Apr 17 '12 at 5:29

I would suggest adding Linear to the list.

I asked a question on math.SE about what I, as an engineer, think of as linear minimum mean square error estimation of a random variable $Y$ given the value of a random variable $X$ (meaning estimating $Y$ as $\hat{Y} = aX+b$ with $a$ and $b$ being chosen so as to minimize $E[(Y-aX-b)^2]$), and gave a partial answer. One of the comments on the question said

"I am somewhat uncomfortable with your language, since I fear that this way of using the word "linear" might feed into the popular misunderstanding that the reason why linear regression in called linear regression is that one is fitting a line. People who think that then find it confusing when a statistician insists that one is doing linear regression when one fits a parabola or a sine wave, etc."

So, what does linear regression mean to a statistician?

-
Related question on the site in reference to this answer, What does linear stand for in linear regression? –  Andy W Jan 12 '12 at 14:01
@AndyW So would you say that Linear belongs on the list that Peter Flom started, or not? –  Dilip Sarwate Jan 12 '12 at 14:19
yes I agree it fits the bill for this list. (+1) –  Andy W Jan 12 '12 at 14:48
It fits the list, but for an unusual reason: the meaning of the term "linear" is well established and used consistently across many mathematically-oriented fields. The potential confusion concerns which part of the formula is linear. –  whuber Jan 12 '12 at 21:16

probability

It seems to me that most of the problems associated with interpreting hypothesis tests and confidence intervals stem from the application of a Bayesian definition of "probability" when the procedure is based on a frequentist one. For example the p-value being the probability the null hypothesis is true, when AFAICS no probability can be associated with the truth of a particular hypothesis in a frequentist setting.

-
It looks like the same consideration would apply for those who are used to say that the (true) parameter has a 95% chance of lying between xx and xx, when speaking of/interpreting confidence intervals. –  chl Jan 12 '12 at 15:45
yes absolutely! –  Dikran Marsupial Jan 12 '12 at 15:51
+1 I would phrase your last sentence slightly differently, however. Within the frequentest setting the probability that the null hypothesis is true is either 1 or 0, but you don't know which. (Strictly speaking, this is not quite right, because 'probability' is a long-run relative frequency and 'long-run frequency' doesn't really apply. Nonetheless, when phrased this way people can understand what's being said / how we understand the situation we're in. For example, people realize that you can't use the null hypothesis p-value as the probability that the null hypothesis is true.) –  gung Jan 12 '12 at 17:18
"because 'probability' is a long-run relative frequency" Lots of probabilists will vehemently dispute that statement –  Dilip Sarwate Jan 12 '12 at 18:31

"Confidence"

It's very hard to dissuade non-statisticians that their confidence interval is not (directly) a statement about the credibility of different parameter values.

To have confidence, in the technical meaning of the term, we need to imagine some set of repeated experiments, each one computing an interval in some pre-specified way. To be a 95% confidence interval, 95% of these uses of the formula will trap the relevant parameter of interest.

But non-statisticians routinely interpret "95% confidence" to be a statement about plausible parameter values, based on one experiment alone. Typically, they assume that the interval covers 95% of some posterior beliefs about the parameter, i.e. "we're pretty certain the parameter is between $a$ and $b$". This instead defines a credible interval.

(There are of course situations when the two notions agree, either approximately or exactly. But in general they don't, and numerical agreement doesn't remove the problem of misuse of technical terms.)

-

Error.

In statistics, an "error" is a deviation of an actual data value from the prediction of a model.

In real life, an error is a spllng mstake or other goof.

-
Isn't a spelling mistake just a deviation from the actual (intended) value of the communication medium? I don't really see how this is a different word, it's just that it's used in a different (but not conflicting) context. I find it hard to believe that it would be all that confusing to someone new to the field. –  naught101 Apr 17 '12 at 5:21
One reason why a value can differ from a prediction is that the experimenter messed up. That is like a spelling mistake. But why is your weight different than the average weight of all people of your gender and age? Why is your income different than the mean income? In statistics, this deviation from a mean is an "error" but it isn't a mistake, just variation. –  Harvey Motulsky Apr 17 '12 at 18:50
True, but it depends how you look at it, I think. if you look at the spelling of a word over a sample of a primary school, you're going to get variation, caused by humans, yes, but nor caused by the experimenter. You could look at the same thing over writing in english from different ages. I reckon you'd find that early english had a lot higher variability :) –  naught101 Apr 18 '12 at 0:11

"Likelihood" -- it is synonymous with "probability" in everyday speech, but in Statistics it has a special meaning: it is a function of the parameters of a statistical model, whose value is the probability of the observed outcome assuming that the parameters are equal to the parameter values.

-

To us (or at least me), "randomness" of a "sample" suggests that it is representative of the "population".

To others, "randomness" sometimes implies that a person/thing is unusual.

-
I haven't run into that confusion over "randomness". But if you have, then clearly it exists. –  Peter Flom Jan 19 '12 at 10:38
More precisely, that it has existed –  Thomas Levine Jan 19 '12 at 13:28
The latter usage of "random" seems pretty recent to me. I find it a bit annoying for that reason (just makes stats harder for people to understand). It's even more annoying when I hear myself using it in that sense.. –  naught101 Apr 17 '12 at 5:23

"Inference"

One of the hardest things for me to understand at first was the difference between a population and a sample. Statisticians write these fancy population level regression equations and then all of a sudden drop down into sample level work and the $\beta$s become $b$s. It took me a long time to realize that you were using the sample level data and regression equations to estimate the population level parameters.

Another important part about inference is the central limit theorem. Once you realize that you are simply sampling from a population -- although sampling is another complicated feature akin to inference -- then you understand that even if the sample mean holds one value, that value isn't necessarily the same mean as in the population.

Perhaps I took a relatively loose understanding of your question, but once someone understands inference or the differences between a sample and the population then the entirety of statistics opens to them.

-

Ecological, commonly used to refer to biological systems, but also a statistical fallacy. From Wikipedia:

An ecological fallacy (or ecological inference fallacy) is an error in the interpretation of statistical data in an ecological study, whereby inferences about the nature of specific individuals are based solely upon aggregate statistics collected for the group to which those individuals belong. This fallacy assumes that individual members of a group have the average characteristics of the group at large.

-

I think one should distinguish between terms confusing the public and terms confusing statisticians. The above suggestions, are mostly terms well understood by statisticians and (possibly) misunderstood by the public. I wish to add to the list some terms miss understood by statisticians:

• Bayesian: Originally referred to what is now known as subjective Bayes (a.k.a. epistemic, De-Finetti). Today the term will be used anytime Bayes rule shows up, rarely in the context of subjective beliefs, which is considered decision-theory.
• Empirical Bayes: Originally referring to a frequentist setup with a non parametric prior. Today, will typically mean the parameters of the parametric (objective) prior are estimated and not known a-priori. I.e., what was once known as type-II maximum likelihood.
• Non parametric: Sometimes refers to "model free". Sometimes to "distribution free". Has become practically uninformative in the days "parametric" models might include millions of parameters.
• Type III error: sometimes referring to a sign error. Sometime referring to a misspecification of the model.
-
When I asked, I intended "terms confusing to the general public" but certainly terms confusing to statisticians are also worth listing –  Peter Flom Jan 19 '12 at 10:36
This probably should be split up into separate answers. –  naught101 Apr 17 '12 at 4:08

Is a "survey" a type of math ("survey sampling") or a piece of paper ("questionnaire")?

I haven't conducted a survey on this, but I suspect that much of the public considers a "survey" to be the latter. I suspect further that they don't think about the former.

-
Aren't surveys done by surveyors? ;) –  zbicyclist Jan 18 '12 at 5:42

I usually find people being quite ad-hoc when using them, employing them interchangeably without first explicitly defining what they mean and I have actually come across papers that refer to "loading vectors" and sometimes mean the PCs themselves and other times the "weights" associated with a specific PC.

Probably the fact that Jollifee's excellent reference on Principal Components states at the end of section 1.1 "Some authors distinguish between the terms ‘loadings’ and ‘coefficients,’ depending on the normalization constraint used, but they will be used interchangeably in this book." just made people think they have a free pass to mix and match terminology to their liking....

-

Additive model. Still not really sure what this means. I think it refers to a model without interaction terms. But then I will come across an article where they're using it to refer to something else, i.e. a spline model.

-

"Statistics"

To the general public, a substitute for, "now I'm about to lie to you and speak in a way you don't understand."

-