53
$\begingroup$

We statisticians use many words in ways that are slightly different from the way everyone else uses them. This cause lots of problems when we teach or explain what we are doing. I'll start a list (and now I'll add some definitions, per comments):

  • Power is the probability of correctly reject a false null hypothesis in a hypothetical situation where the data comes from a specific alternate hypothesis or range of alternates. Usually, this means "our statistical method should succeed" if "something is happening".
  • Bias - a statistic is biased if it is systematically different from the population parameter associated with it.
  • Significance - results are statistically significant at some percent (often 5%) in the following situation: If the population which the sample comes from has a true effect of 0, a statistic at least as extreme as the one gotten from the sample would only occur 5% of the time.
  • Interaction - Two independent variables interact if the relationship between the dependent variable and one independent variable is different at different levels of the other independent variable

But there have to be many others!

Improve this question
$\endgroup$
8
  • 7
    $\begingroup$ I would suggest people add some greater context to their answers as well. Examples might be the same word used differently in different fields (fixed effects Gelman, 2005) or words that have different meanings in different contexts (significance vs. statistical significance Bushway et al., 2006). $\endgroup$
    – Andy W
    Commented Jan 12, 2012 at 13:02
  • 7
    $\begingroup$ It would be good if answerers could explain in one or two sentences what the "technical" term really means, or what might cause it to be perceived as having a different meaning. $\endgroup$
    – chl
    Commented Jan 12, 2012 at 13:02
  • $\begingroup$ I will complete my answer according to your comments later on ;-) $\endgroup$
    – ocram
    Commented Jan 12, 2012 at 13:20
  • 1
    $\begingroup$ ... and "correlation" ! $\endgroup$
    – Stéphane Laurent
    Commented Jan 12, 2012 at 15:59
  • 1
    $\begingroup$ For "sample," see the comments to stats.stackexchange.com/questions/20945/…. $\endgroup$
    – whuber ♦
    Commented Jan 12, 2012 at 16:49

16 Answers 16

Reset to default
23
$\begingroup$

"Significant" is the biggest one I run into, because it has both a common English-use meaning and that meaning will crop up in the discussion of research results. I even find myself mixing in "significant" to mean important in the same sentence where I've talked about statistical results.

That way lies madness.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Right, but there's no better word for "I'm pretty damn sure it's significant, but I haven't done any tests on it, and won't, because it's obvious/can't be done/whatever" $\endgroup$
    – naught101
    Commented Apr 17, 2012 at 5:29
  • 1
    $\begingroup$ The question to keep asking -- myself, yourself and others -- is But what does it signify? $\endgroup$
    – Nick Cox
    Commented Mar 22, 2021 at 9:24
20
$\begingroup$

I would suggest adding Linear to the list.

I asked a question on math.SE about what I, as an engineer, think of as linear minimum mean square error estimation of a random variable $Y$ given the value of a random variable $X$ (meaning estimating $Y$ as $\hat{Y} = aX+b$ with $a$ and $b$ being chosen so as to minimize $E[(Y-aX-b)^2]$), and gave a partial answer. One of the comments on the question said

"I am somewhat uncomfortable with your language, since I fear that this way of using the word "linear" might feed into the popular misunderstanding that the reason why linear regression in called linear regression is that one is fitting a line. People who think that then find it confusing when a statistician insists that one is doing linear regression when one fits a parabola or a sine wave, etc."

So, what does linear regression mean to a statistician?

Improve this answer
$\endgroup$
6
  • 5
    $\begingroup$ Related question on the site in reference to this answer, What does linear stand for in linear regression? $\endgroup$
    – Andy W
    Commented Jan 12, 2012 at 14:01
  • 1
    $\begingroup$ @AndyW So would you say that Linear belongs on the list that Peter Flom started, or not? $\endgroup$
    – Dilip Sarwate
    Commented Jan 12, 2012 at 14:19
  • 1
    $\begingroup$ yes I agree it fits the bill for this list. (+1) $\endgroup$
    – Andy W
    Commented Jan 12, 2012 at 14:48
  • 4
    $\begingroup$ It fits the list, but for an unusual reason: the meaning of the term "linear" is well established and used consistently across many mathematically-oriented fields. The potential confusion concerns which part of the formula is linear. $\endgroup$
    – whuber ♦
    Commented Jan 12, 2012 at 21:16
  • 1
    $\begingroup$ Can you give an example of how you fit a parabola and still call it a linear model? $\endgroup$
    – oneloop
    Commented Mar 27, 2018 at 16:08
15
$\begingroup$

"Confidence"

It's very hard to dissuade non-statisticians that their confidence interval is not (directly) a statement about the credibility of different parameter values.

To have confidence, in the technical meaning of the term, we need to imagine some set of repeated experiments, each one computing an interval in some pre-specified way. To be a 95% confidence interval, 95% of these uses of the formula will trap the relevant parameter of interest.

But non-statisticians routinely interpret "95% confidence" to be a statement about plausible parameter values, based on one experiment alone. Typically, they assume that the interval covers 95% of some posterior beliefs about the parameter, i.e. "we're pretty certain the parameter is between $a$ and $b$". This instead defines a credible interval.

(There are of course situations when the two notions agree, either approximately or exactly. But in general they don't, and numerical agreement doesn't remove the problem of misuse of technical terms.)

Improve this answer
$\endgroup$
14
$\begingroup$

probability

It seems to me that most of the problems associated with interpreting hypothesis tests and confidence intervals stem from the application of a Bayesian definition of "probability" when the procedure is based on a frequentist one. For example the p-value being the probability the null hypothesis is true, when AFAICS no probability can be associated with the truth of a particular hypothesis in a frequentist setting.

Improve this answer
$\endgroup$
4
  • 5
    $\begingroup$ It looks like the same consideration would apply for those who are used to say that the (true) parameter has a 95% chance of lying between xx and xx, when speaking of/interpreting confidence intervals. $\endgroup$
    – chl
    Commented Jan 12, 2012 at 15:45
  • 1
    $\begingroup$ yes absolutely! $\endgroup$
    – Dikran Marsupial
    Commented Jan 12, 2012 at 15:51
  • 1
    $\begingroup$ +1 I would phrase your last sentence slightly differently, however. Within the frequentest setting the probability that the null hypothesis is true is either 1 or 0, but you don't know which. (Strictly speaking, this is not quite right, because 'probability' is a long-run relative frequency and 'long-run frequency' doesn't really apply. Nonetheless, when phrased this way people can understand what's being said / how we understand the situation we're in. For example, people realize that you can't use the null hypothesis p-value as the probability that the null hypothesis is true.) $\endgroup$
    – gung - Reinstate Monica
    Commented Jan 12, 2012 at 17:18
  • 2
    $\begingroup$ "because 'probability' is a long-run relative frequency" Lots of probabilists will vehemently dispute that statement $\endgroup$
    – Dilip Sarwate
    Commented Jan 12, 2012 at 18:31
11
$\begingroup$

"Likelihood" -- it is synonymous with "probability" in everyday speech, but in Statistics it has a special meaning: it is a function of the parameters of a statistical model and a particular data situation, whose value is the probability of the observed outcome assuming that the parameters are equal to the parameter values.

Improve this answer
$\endgroup$
1
  • $\begingroup$ In turn that's a loose statement when applied to continuous variables, as the likelihood then depends on probability density. Yet further, constant factors that shift the likelihood up or down but don't affect the position of the maximum are often omitted. $\endgroup$
    – Nick Cox
    Commented Mar 22, 2021 at 9:27
9
$\begingroup$

Error.

In statistics, an "error" is a deviation of an actual data value from the prediction of a model.

In real life, an error is a spllng mstake or other goof.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Isn't a spelling mistake just a deviation from the actual (intended) value of the communication medium? I don't really see how this is a different word, it's just that it's used in a different (but not conflicting) context. I find it hard to believe that it would be all that confusing to someone new to the field. $\endgroup$
    – naught101
    Commented Apr 17, 2012 at 5:21
  • 2
    $\begingroup$ One reason why a value can differ from a prediction is that the experimenter messed up. That is like a spelling mistake. But why is your weight different than the average weight of all people of your gender and age? Why is your income different than the mean income? In statistics, this deviation from a mean is an "error" but it isn't a mistake, just variation. $\endgroup$
    – Harvey Motulsky
    Commented Apr 17, 2012 at 18:50
  • $\begingroup$ True, but it depends how you look at it, I think. if you look at the spelling of a word over a sample of a primary school, you're going to get variation, caused by humans, yes, but nor caused by the experimenter. You could look at the same thing over writing in english from different ages. I reckon you'd find that early english had a lot higher variability :) $\endgroup$
    – naught101
    Commented Apr 18, 2012 at 0:11
  • $\begingroup$ @HarveyMotulsky: analytical chemistry uses error in both ways. We speak of systematic error, random error and gross errors. Textbook: "gross errors can be avoided". $\endgroup$
    – cbeleites
    Commented May 6, 2019 at 16:15
  • $\begingroup$ Harold Jeffreys pointed out that the primary statistical sense is erratic, not erroneous. Otherwise put, error is uncontrolled variation and we are usually ignorant about any true value existing independently of the data. $\endgroup$
    – Nick Cox
    Commented Mar 22, 2021 at 9:30
9
$\begingroup$

"Inference"

One of the hardest things for me to understand at first was the difference between a population and a sample. Statisticians write these fancy population level regression equations and then all of a sudden drop down into sample level work and the $\beta$s become $b$s. It took me a long time to realize that you were using the sample level data and regression equations to estimate the population level parameters.

Another important part about inference is the central limit theorem. Once you realize that you are simply sampling from a population -- although sampling is another complicated feature akin to inference -- then you understand that even if the sample mean holds one value, that value isn't necessarily the same mean as in the population.

Perhaps I took a relatively loose understanding of your question, but once someone understands inference or the differences between a sample and the population then the entirety of statistics opens to them.

Improve this answer
$\endgroup$
8
$\begingroup$

To us (or at least me), "randomness" of a "sample" suggests that it is representative of the "population".

To others, "randomness" sometimes implies that a person/thing is unusual.

Improve this answer
$\endgroup$
3
  • 2
    $\begingroup$ I haven't run into that confusion over "randomness". But if you have, then clearly it exists. $\endgroup$
    – Peter Flom
    Commented Jan 19, 2012 at 10:38
  • 3
    $\begingroup$ More precisely, that it has existed $\endgroup$
    – Avery Richardson
    Commented Jan 19, 2012 at 13:28
  • 2
    $\begingroup$ The latter usage of "random" seems pretty recent to me. I find it a bit annoying for that reason (just makes stats harder for people to understand). It's even more annoying when I hear myself using it in that sense.. $\endgroup$
    – naught101
    Commented Apr 17, 2012 at 5:23
5
$\begingroup$

I think one should distinguish between terms confusing the public and terms confusing statisticians. The above suggestions are mostly terms well understood by statisticians and (possibly) misunderstood by the public. I wish to add to the list some terms misunderstood by statisticians:

  • Bayesian: Originally referred to what is now known as subjective Bayes (a.k.a. epistemic, De Finetti). Today the term will be used anytime Bayes' rule shows up, rarely in the context of subjective beliefs, which is considered decision theory.
  • Empirical Bayes: Originally referring to a frequentist setup with a non parametric prior. Today, will typically mean the parameters of the parametric (objective) prior are estimated and not known a priori. I.e., what was once known as type-II maximum likelihood.
  • Non parametric: Sometimes refers to "model free". Sometimes to "distribution free". Has become practically uninformative in the days "parametric" models might include millions of parameters.
  • Type III error: sometimes referring to a sign error. Sometime referring to a mis-specification of the model.
Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ When I asked, I intended "terms confusing to the general public" but certainly terms confusing to statisticians are also worth listing $\endgroup$
    – Peter Flom
    Commented Jan 19, 2012 at 10:36
  • $\begingroup$ This probably should be split up into separate answers. $\endgroup$
    – naught101
    Commented Apr 17, 2012 at 4:08
4
$\begingroup$

Ecological, commonly used to refer to biological systems, but also a statistical fallacy. From Wikipedia:

An ecological fallacy (or ecological inference fallacy) is an error in the interpretation of statistical data in an ecological study, whereby inferences about the nature of specific individuals are based solely upon aggregate statistics collected for the group to which those individuals belong. This fallacy assumes that individual members of a group have the average characteristics of the group at large.

Improve this answer
$\endgroup$
1
  • $\begingroup$ The main use I have seen for this is in discussions of social data, focusing on the difference between individual data and areal data, and not in ecology! $\endgroup$
    – Nick Cox
    Commented Mar 22, 2021 at 9:34
4
$\begingroup$

"Loadings", "Coefficients" and "Weights"; when talking about Principal Component Analysis.

I usually find people being quite ad hoc when using them, employing them interchangeably without first explicitly defining what they mean and I have come across papers that refer to "loading vectors" and sometimes mean the PCs themselves and other times the "weights" associated with a specific PC.

Probably the fact that Jolliffe's excellent reference on Principal Components states at the end of section 1.1 "Some authors distinguish between the terms ‘loadings’ and ‘coefficients,’ depending on the normalization constraint used, but they will be used interchangeably in this book." just made people think they have a free pass to mix and match terminology to their liking....

Improve this answer
$\endgroup$
3
$\begingroup$

Is a "survey" a type of math ("survey sampling") or a piece of paper ("questionnaire")?

I haven't conducted a survey on this, but I suspect that much of the public considers a "survey" to be the latter. I suspect further that they don't think about the former.

Improve this answer
$\endgroup$
1
  • 2
    $\begingroup$ Aren't surveys done by surveyors? ;) $\endgroup$
    – zbicyclist
    Commented Jan 18, 2012 at 5:42
1
$\begingroup$

Additive model. Still not really sure what this means. I think it refers to a model without interaction terms. But then I will come across an article where they're using it to refer to something else, i.e. a spline model.

Improve this answer
$\endgroup$
1
$\begingroup$

Consistency

First, many other people read into this a notion of something like "does not have any (internal) contradictions", which is related, but surely not equivalent, to definitions used in statistics.

Second, even within statistics, it has more than one meaning, such as consistency of an estimator, consistency of a test or consistent model selection criteria.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ Consistency is also (uncommonly but occasionally) a term of art in circular statistics, where it measures how far a set of directions on the circle are similar. Consistency 1 means all directions are equal. In that context it is in my view a good term, although less common than other more awkward or more cryptic terms such as mean resultant length or vector strength. $\endgroup$
    – Nick Cox
    Commented Mar 8, 2023 at 11:21
0
$\begingroup$

One of the terms that I find most confusing is the "confusion matrix". Of course, The term used itself is confusing, not the concept.

I tried to track the history of the term and it is quite interesting too. The confusion matrix was invented at 1904 by (http://en.wikipedia.org/wiki/Karl_Pearson). He used the term http://en.wikipedia.org/wiki/Contingency_table. It appeared at Karl Pearson, F.R.S. (1904). Mathematical contributions to the theory of evolution (PDF). Dulau and Co. http://ia600408.us.archive.org/18/items/cu31924003064833/cu31924003064833.pdf

During War World 2, https://en.wikipedia.org/wiki/Detection_theory was developed as investigation of the relations between stimulus and responds. The confusion matrix was used there.

Due to detection theory, the term was used a psychology. From there the term reached machine learning.

It seems that though the concept was invented in statistics, a field very related to machine learning, it reached machine learning after a detour in during a period of 100 years.

For some references of the usage of the term see: What is the origin of the term confusion matrix?

Improve this answer
$\endgroup$
-4
$\begingroup$

"Statistics"

To the general public, a substitute for, "now I'm about to lie to you and speak in a way you don't understand."

Improve this answer
$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.