116
$\begingroup$

Statistics is everywhere; common usage of statistical terms is, however, often unclear.

The terms probability and odds are used interchangeable in lay English despite their well-defined and different mathematical expressions.

Not separating the term likelihood from probability routinely confuses physicians trying to quantify the probability of breast cancer given a positive mammography, “Oh, what nonsense. I can’t do this. You should test my daughter; she is studying medicine.”

Equally spread is the use of correlation instead of association. Or correlation implying causation.

In Al Gore's famous documentary An Inconvenient Truth, a slide illustrates the correlation of ice core $\small \text{CO}_2$ and temperatures, leaving the more technical work to prove causation out of the discussion:

enter image description here

QUESTION: Which statistical terms pose interpretation problems when used without mathematical rigor, and are, therefore, worth correcting?

$\endgroup$
5
  • 4
    $\begingroup$ Odds vs. probability among laymen doesn't seem to be a problem to me since laymen wouldn't be calculating them anyway, they would just be saying the values are low or high, and the two are directly correlated. $\endgroup$
    – user541686
    Mar 21, 2016 at 23:51
  • 2
    $\begingroup$ @Mehrdad I agree. Actually, this is the point... is there any situation where misuse of these words, which have been adopted and technified within the confines of statistics, results in problems. For instance, it is clear that there is an important body of research behind climate change, but in many other circumstances false claims can be made by suggesting that correlation equals causation. In the case of odds and probabilities, either can be converted into the other, so the only risk is misunderstanding your bets. $\endgroup$ Mar 22, 2016 at 0:51
  • 5
    $\begingroup$ @Mehrdad The point about odds is an interesting one, but I think it's more complicated than meets the eye. When laymen talk about odds, they usually mean gambling odds, and these are very often expressed in the "odds against" format. So in the system that most people are familiar with, a high value for odds is associated with a low probability, though for a statistician high odds are associated with high probability. This is therefore quite ripe for confusion: see also our post on Odds Made Simple $\endgroup$
    – Silverfish
    Mar 23, 2016 at 12:06
  • 9
    $\begingroup$ It's probably worth bearing in mind that some of these terms were pre-existing in the English language (with loose meaning), prior to being appropriated by statistics and given rigorous technical definitions. It's a little condensing to take the word, change the meaning, and then run around blaming others for using it wrong when they're just using it with the older, non-technical, definition. $\endgroup$
    – R.M.
    Mar 23, 2016 at 14:34
  • $\begingroup$ I really don't like calling tests "post hoc" even when they are planned in advance. I think this began with some stat package but it is now pervasive. $\endgroup$
    – David Lane
    Mar 8, 2017 at 15:14

20 Answers 20

122
$\begingroup$

It can be futile to fight against shifts in language. But

parameter does not mean variable

In classical statistics, which in this case starts precisely with R.A. Fisher who first used the term with this meaning, a parameter is an unknown constant to be estimated, say a population mean or correlation. In mathematics, there are related but not identical meanings, as when a curve is given parametrically. In many sciences, parameter is just another word for a measure (itself a term dense with mathematical meaning), property or variable, say length or conductivity or porosity or virtue, as the case may be. Naturally, an individual's length or virtue is unknown before it is measured. but statistically minded people can be bemused by its use for a set of such measurements. In ordinary or vulgar parlance, parameters (almost always plural) often mean the limits of something, say a personal relationship or a political policy, perhaps stemming from some original confusion with perimeter. With high prior probability it is to be presumed that Bayesians will speak for themselves on their own usages (grateful nod to @conjugateprior).

skewed does not mean biased

For a century or more, skewness has had a specific statistical sense of referring to asymmetry of distributions, whether assessed graphically, measured numerically, or presumed theoretically as a matter of faith or hope. For much longer, or so it may be guessed, bias has meant being wrong on average, which -- so long as we know the truth, meaning a true or correct value -- can be quantified as systematic error. Skewed in ordinary language has a common sense of being warped or distorted, and thus of being incorrect, wrong and so also biased too. That sense (so far as I have noticed, only quite recently) has begun filtering back into statistical discussions, so that the original meaning of skewness is in some danger of being blurred or submerged.

correlation does not mean agreement

Correlation has attracted several precise senses in statistics, which have in common an idea of a bivariate relationship perfect in some precise sense: the leading cases are linear and monotone relationship. It is often diluted, even in statistical discussions, to mean almost any kind of relationship or association. What correlation does not mean, necessarily, is agreement: thus $y = a + bx$ implies Pearson correlation of $1$ or $-1$ so long as $b \ne 0$, but agreement $y = x$ requires the very strict condition $a =0, b= 1$.

unique does not mean distinct

It is quite common to talk about the distinct values of data as unique, but unique is still ideally better preserved as meaning occurring once only. My own guess is that some of the blame stems from the Unix [sic] utility uniq and its imitators, which reduce possibly repeated values to a set in which each value really is unique. The usage, on this guess, conflates input and output of a program. (Conversely, if we talk of duplicates in data, we rarely restrict ourselves to doubletons that occur precisely twice. The term replicates would make more sense linguistically but has been pre-empted for deliberate replication of controls in experiments; the resulting response values are usually not at all identical, which is much of the point.)

samples are rarely repeated

In statistics, a sample includes several values, and repeated sampling is a high theoretical virtue, but one rarely practised, except by simulation, which is our customary term for any kind of faking in silico. In many sciences, a sample is a single object, consisting of a lump, chunk or dollop of water, soil, sediment, rock, blood, tissue, or other substances varying from attractive through benign to disgusting; far from being exceptional, taking many samples may be essential for any serious analysis. Here every field's terminology makes perfect sense to its people, but translation is sometimes needed.

error does not usually mean mistake; as Harold Jeffreys pointed out, the primary sense is erratic, not erroneous.

Nevertheless, we should be wary of our own sins or quirks of terminology:

expected values or expectations (for means over the possible outcomes) may not be what you expect at all, and could even be impossible: in tossing a die fairly with outcomes 1 to 6, the expected value is 3.5

regression is not going backwards

stationary does not mean immobile or fixed

confidence has nothing to do with anyone's mental or psychological state

significance has only sometimes its everyday meaning

exact is often an honorific term, referring to a conveniently tractable solution or calculation rather than one appropriate to the problem

right-skewed distributions to many look skewed left, and vice versa (and the terminology of right and left for skewness assumes that you are looking at something like a conventional histogram, with horizontal magnitude axis)

the lognormal is so called because it's an exponentiated normal

but the lognormal is more normal than the normal

the Gaussian was discovered by De Moivre

Poisson didn't discover the Poisson, let alone Poisson regression

the bootstrap won't help you with your footwear

the jackknife doesn't cut

kurtosis is not a medical condition

stem-and-leaf plots don't refer to plants

a dummy variable is useful, not pointless or stupid

who on Earth (or anywhere else) thinks that heteroscedasticity is really a preferable term over unequal variability?

robust now has at least two major technical meanings for different groups, neither of which inhibits its frequent use, even in technical discussions, to mean merely something like "asserted to behave well"

IV now has at least two major meanings for different groups

factor now has at least two major meanings for different groups

normalize and standardize have uncountably many meanings (we really need to standardize there)

versus describing a graph means vertical variable versus horizontal variable, unless it means the opposite

and (last but not least, to coin a phrase) statistics has at least three major meanings.

Notes:

  1. Despite any appearances to the contrary, I think this is a good, serious question.

  2. Fashions shift. Well into the twentieth century, it seems that many people (no names, no pack-drill, but Karl Pearson could be mentioned) could only invent terms by reaching for their Greek and Latin dictionaries. (It would be unfair not to give him credit for scatter plot.) But R.A. Fisher did hijack many pre-existing English words, including variance, sufficiency, efficiency and likelihood. More recently, J.W. Tukey was a master in using homely terms, but few should feel distress that sploms and badmandments did not catch on.

  3. One comment is based on recollection of "Life is [...] Multiplicative rather than additive: the log normal distribution is more normal than the normal." Anon. 1962. Bloggins's working rules. In Good, I.J. (Ed.) The scientist speculates: an anthology of partly-baked ideas. London: Heinemann, 212-213 (quotation on p.213).

$\endgroup$
8
  • $\begingroup$ Comments are not for extended discussion; this conversation has been moved to chat. $\endgroup$
    – whuber
    Mar 22, 2016 at 20:05
  • 2
    $\begingroup$ Heteroscedasticity totally rocks the cat box! "Unequal variability?" [Phuagh!] ) (+1 very good otherwise ;) $\endgroup$
    – Alexis
    Mar 23, 2016 at 21:59
  • 3
    $\begingroup$ May be worth adding that regression testing is frequently used in context of software development, where, broadly speaking, it refers to going backwards. $\endgroup$
    – Konrad
    May 16, 2018 at 6:53
  • $\begingroup$ @Konrad Interesting, but then (correct me if I'm wrong) (a) that wouldn't be a misuse of the word and (b) the word there doesn't have a statistical sense. $\endgroup$
    – Nick Cox
    May 16, 2018 at 6:58
  • 1
    $\begingroup$ "the bootstrap won't help you with your footwear" made my day :) :) $\endgroup$ Nov 11, 2018 at 17:55
35
$\begingroup$

Some of the things I encounter:

  1. Treating significance level and CI coverage probabilities as interchangeable, so that people end up doing things like speaking of "95% significance".

    [What's worse is when people who make such errors point to their lecture notes -- or even textbook -- as support for this; in other words the mistake is not theirs, but is being compounded a hundredfold or many-thousands-fold, and worse, even if they understand it correctly, they may actually have to repeat the error anyway, to pass the subject.]

  2. There's also a common tendency to think that "significance" somehow exists outside a specific hypothesis/question (leading to questions like "are my data significant" without any clear notion of what question is to be addressed). [A related issue is the "what test should I use for these data?" as if it were the data - rather than the question to be answered - that's the driver of choice of analysis. (While the "design" of the study can impact the specific tests used, the question of interest is more important -- for example, if you have three groups available but your question of interest only relates to a comparison of two of them, the fact that you have three doesn't force you to do a one-way type analysis rather than a straight comparison of the two groups of interest ... as long as your choice of analysis doesn't derive from what the data show. Ideally you plan your questions and analyses before you have data, rather than throwing analysis at data and see what sticks, which it seems post-hoc analysis questions - including "what test should I use for these data?" - tend to lead to.)

  3. An occasional tendency to refer to the complement of p-value as some sort of "confidence in", or "probability of" the alternative.

  4. "nonparametric data"; another one unfortunately found in a couple of books (and, sadly, in an article that purports to correct a common error) this one comes up so often that it's in my short list of automatically generated comments (which begins "Data are neither parametric nor nonparametric; those are adjectives that apply to models or techniques...") (thanks Nick Cox for reminding me of this particular bugbear)

    Usually what is intended is "non-normal data" but parametric doesn't imply normal, and having approximate normality doesn't imply we need parametric procedures. Similarly, non-normality doesn't imply we need non-parametric procedures. Occasionally, what is intended is "ordinal data" or "nominal data" but in neither case does that imply that finite-parametric models are inappropriate.

  5. A common tendency to misunderstand the meaning of "linear" in "linear model" in a way that would be inconsistent with the use of the term "linear" in "generalized linear model". This is partly the fault of the way we use terminology.

  6. conflating the mean-minus-median kind of skewness with third moment skewness, and conflating a zero in either (or even both) with symmetry. Both errors are frequently found in basic texts widely used in some particular application areas. [There's a related error of treating zero skewness and zero excess kurtosis as implying normality]

  7. this one is so common it's becoming hard to call it an error any more (due in part to the efforts of a particular program) -- calling excess kurtosis simply "kurtosis"; a mistake pretty much guaranteed to lead to communication problems.

$\endgroup$
16
  • 2
    $\begingroup$ +1. I want to remind you of the grotesque "non-parametric data", which belongs better in this list than in mine. Excess kurtosis is an ugly sibling of crude kurtosis. $\endgroup$
    – Nick Cox
    Mar 22, 2016 at 2:43
  • $\begingroup$ @Nick Thanks, I have been sitting here staring at my list saying "there's something else that really annoys me that I know belongs here". That's the one. $\endgroup$
    – Glen_b
    Mar 22, 2016 at 2:56
  • 3
    $\begingroup$ Another one is "statistical test" expanded so widely that it becomes the opening question: what test should I apply to my data? often in the belief that there will be a single answer of the form "Student's t", "Mann-Whitney" or "chi-square". To which my answer is more usually, perhaps none at all, or we have to look carefully at your data and discuss what your real question is before we can think about that. $\endgroup$
    – Nick Cox
    Mar 22, 2016 at 3:02
  • $\begingroup$ @nick That one closely relates to my item 2. I wonder if there's a good way to expand that one. $\endgroup$
    – Glen_b
    Mar 22, 2016 at 3:03
  • 1
    $\begingroup$ I fear that many statistical texts (appear to) encourage such thinking. $\endgroup$
    – Nick Cox
    Mar 22, 2016 at 3:11
34
$\begingroup$

"Data" is plural. (The singular is "datum").

$\endgroup$
8
  • 2
    $\begingroup$ Do you really talk about a datum? More usually, that point... that value ..., that observation..., at least hereabouts. $\endgroup$
    – Nick Cox
    Mar 22, 2016 at 0:00
  • 5
    $\begingroup$ Data is also a singular android, that assimilates data about humans he observes to come to data driven Data conclusions, often to humorous effect. $\endgroup$ Mar 22, 2016 at 0:00
  • 6
    $\begingroup$ Plural data requires not only agreement of the verb - "data are" rather than "data is", but of quantifiers - "many data" rather than "much data", "fewer data" rather than "less data". So few people manage to be consistent that it seems to be a lost cause. $\endgroup$ Mar 22, 2016 at 14:52
  • 6
    $\begingroup$ Despite years (nay decades) of fighting this (my Latin teachers would be pleased), I have come to a view similar to @Scortchi's. But I try to use the word dataset where possible, influenced particularly by StataCorp practices. That solves some of the difficulties. $\endgroup$
    – Nick Cox
    Mar 22, 2016 at 15:36
  • 3
    $\begingroup$ For singular I'd go with "data point" instead of "datum"... $\endgroup$
    – user541686
    Mar 25, 2016 at 1:36
18
$\begingroup$

Percent vs. Percentage Points: If something increases from 1% to 2%, it increased by 100%. Or: you can say that it increased by 1 percentage point.

Stating that the increase was 1% is very misleading.

$\endgroup$
17
$\begingroup$

While not strictly a statistical term, I vote to retire endogeneity. It's used to refer to everything from reverse causation through confounding to selection and collider bias, when all people really want to do is say: "That effect is not identified".

$\endgroup$
1
  • $\begingroup$ Comments are not for extended discussion; this conversation has been moved to chat. $\endgroup$
    – Glen_b
    Mar 25, 2016 at 8:46
16
$\begingroup$

"Regression towards the mean" does not mean that if we've observed a certain number of iid samples below expected value, the next iid samples are likely to be above the expected value.

$\endgroup$
1
  • 5
    $\begingroup$ +1 This is important. Notable people have been extraordinarily confused by this. For instance, Peter Bernstein's popular book on risk analysis, Against the Gods. characterizes regression to the mean in many different ways--not a single of them correct. $\endgroup$
    – whuber
    Mar 23, 2016 at 21:22
12
$\begingroup$

Kurtosis does not measure "peakedness."

By definition, it is the expected value (average) of $Z^4$.* Thus, $|Z|$-values less than 1 (corresponding to data values within one standard deviation of the mean, where any "peak" would be) contribute very little to kurtosis; nearly all the contribution to kurtosis is from $|Z|$-values greater than 1 (outliers in particular). See http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4321753/ , Figures 2 and 3 in particular.

* Subtract 3 or not; it makes no difference to this point.

$\endgroup$
5
  • 1
    $\begingroup$ This seems more a misunderstood concept than a misused term. (My own answer is a little digressive too, so I don't regard that as a strong criticism.) Independently of that, defining $Z$ would make the answer self-contained, even though it's unlikely that people will have heard of kurtosis but not recognise the allusion to $Z$. $\endgroup$
    – Nick Cox
    Mar 23, 2016 at 21:45
  • 1
    $\begingroup$ I had a Greek-Cypriot stats professor, who taught us that leptokurtic, in Greek, means "narrow-shouldered" or "hunch-backed". Thus, a leptokurtic distribution (e.g., a Laplace or double-exponential) has less mass than the Gaussian (of equal variance) in its "shoulder" areas -- and correspondingly more mass in the head and tail areas. Conversely, a platykurtic distribution (e.g., the uniform) has more mass in the shoulders, and less mass in the head and tail areas, than the normal. $\endgroup$
    – Mico
    Mar 24, 2016 at 19:11
  • 2
    $\begingroup$ Good explanation of the words, but in reality they have nothing to do with the kurtosis statistic that Pearson developed. Pearson had it wrong, but by using those fancy-sounding Greek words he made others think he was onto something profound. His error has harmed statistics education and literacy for over 100 years, sadly. See my paper for pointy ("lepto") distributions where the kurtosis is small, and flat-topped ("platy") distributions where the kurtosis is near infinite. Pearson's kurtosis tells you nothing about "lepto" or "platy". ncbi.nlm.nih.gov/pmc/articles/PMC4321753 $\endgroup$ Mar 25, 2016 at 12:22
  • 1
    $\begingroup$ So what is the appropriate correction to the unrigorous use of kurtosis? I'd say: Remove the concept from introductory classes, and emphasize quantiles instead. $\endgroup$
    – Matt F.
    Aug 1, 2023 at 15:00
  • $\begingroup$ The correction is to define it right. Just call it a measure of how outlier-prone (or extreme value prone) the distribution is. Certainly it is appropriate to talk about outliers in an introductory class, and quantiles don't quite do for that purpose. $\endgroup$ Aug 1, 2023 at 19:14
9
$\begingroup$

I find abbreviations that aren't clearly indicated are a real problem. For example, I see things like GLM and nowhere is it specified if this means general linear model or generalized linear model. Once can usually figure out what is being referenced after digging into the context, but I find this is particularly troublesome for students just starting to learn about statistical models.

Another example of this is IV. Does this mean instrumental variable or independent variable? Often times it's not made clear until you examine the context.

Something else I see confusion over are "moderator" and "interaction." Also, population (as in the population in general) and the population of interest seems to confuse new students unless it is made very clear.

$\endgroup$
6
  • 5
    $\begingroup$ I've also seen GLM used to mean "Global Linear Models" by some in the machine learning crowd. Just to add to the confusion on an already overloaded term $\endgroup$
    – Glen_b
    Mar 22, 2016 at 5:13
  • 1
    $\begingroup$ I partly support this answer/observation. I think "Generalized" (whatever it is) should be better abbreviated to Gz, not to G. Such as GzLM (generalized linear model). $\endgroup$
    – ttnphns
    Mar 22, 2016 at 8:47
  • 3
    $\begingroup$ @ttnphns: some of us write generalised with an s $\endgroup$
    – Henry
    Mar 22, 2016 at 15:48
  • $\begingroup$ I'm curious @ttnphns, what part of this answer don't you support and why? It's quite possible I have a misunderstanding of something, so I'd like to know more if you have anything to offer further. Thanks! $\endgroup$ Mar 22, 2016 at 15:52
  • 1
    $\begingroup$ Huh, I thought IV meant in vitro. =P $\endgroup$
    – user541686
    Mar 25, 2016 at 1:40
8
$\begingroup$

One that is common in everyday language:

average

To the average person out there (bitter irony fully intended), the mean, median, mode and expected value of anything seem to be the same. They have a natural tendency to do a point estimation, with the unconscious and unassailable assumption that there is an underlying normal distribution. And the equally unconscious assumption of a very small variance. The belief that such an estimation 1) exists and 2) will be very useful for them, because they can take it as a practically certain predictor, is so ingrained, that it is basically impossible to convince them otherwise.

For a real-world example, try to talk to a cook who is asking "what is the average size potato", absolutely certain that if you tell him a number, he will be able to use this one for any recipe which specifies a number of potatoes, and have it come out perfect every time. And getting angry at you for trying to tell him "there is no such number". Sadly, it happens in situations with much higher stakes than making soup.

$\endgroup$
8
  • 3
    $\begingroup$ I think this is a little exaggerated. For example, millions if not billions of people seem to have little difficulty with averages in sports. $\endgroup$
    – Nick Cox
    Mar 22, 2016 at 10:08
  • 1
    $\begingroup$ @NickCox it is certainly context dependent. Especially the calculation of an arithmetic mean for given data is unproblematic. I have seen the trouble specifically in the cases I described, where they need a point estimate and assume that the "average" is a very precise one. Also, they assume this "average" to be calculated as a mean, but if you ask them to explain what they mean by average, they roughly describe a mode. $\endgroup$
    – rumtscho
    Mar 22, 2016 at 11:21
  • $\begingroup$ @rumtscho , you're right. Joe Average may tend to think of average as being the mode, or typical. $\endgroup$ Mar 22, 2016 at 16:47
  • $\begingroup$ When people talk about "average" house prices in the UK, they can't tell me the type of average they are using, or if outliners have been excluded. $\endgroup$ Mar 23, 2016 at 19:28
  • 1
    $\begingroup$ There's nothing that says means can't be computed for multimodal distributions, it's just that often, it's not the greatest measure for describing the distribution. Also, I'm not sure it'll do great things for the image of statisticians to tell everyone "You don't know what the word average means!" and then when they point to a dictionary definition, we reply "Well, neither does the dictionary!" $\endgroup$
    – Cliff AB
    Mar 25, 2016 at 16:57
8
$\begingroup$

Linear means:

  • Line-like. As in $y = a + bx$ from elementary algebra. In this respect nonlinear means things like $y = a + bx + cx^{2}$, and also things like $y = ax^{b}$

  • Linear in the parameters being estimated. As in a regression model (linear, logistic, GLM, etc.) entails a sum of products of scalar parameters and independent variables. In this respect nonlinear means things like $y = \frac{e^{a+bx}}{1+e^{a+bx}}$, and $y = a + bx +x\max (x-\theta,0)$.

  • Linear meaning the opposite of dynamic. As in whatever a dependent variable is a function of, it is not a function of its own previous values. In this respect nonlinear means things like $y_{t} = a + by_{t-1} + cx$, and $y_{t}-y_{t-1} = a + b(y_{t-1} - x_{t-x}) + c(x_{t}-x_{t-1}) + dx_{t-1}$.

Where $y$ is a dependent variable, $x$ is an independent variable, and $a, b, c, d$, and $\theta$ are parameters in all the above examples.

$\endgroup$
8
$\begingroup$

Fixed effects and random effects can mean different things for different people. In econometrics fixed effects are actually random and when you think about it every effect in statistics is random so naming something random does not give any meaningful additional information.

$\endgroup$
6
$\begingroup$

The question was about uses of statistical terms that we should CARE to correct. I have been correcting my millennial kids' use of the term 'random' to mean things that are the opposite of random for 10 years now. Considering how many of my trainees struggle to produce a random data sample that is actually random, which happened even before this use of the word, the obfuscation of this term in everyday slang is a crisis.

From the OnlineSlangDictionary:

Definition of random


random

adjective
  • unexpected and surprising.
    All of the sudden this guy jumped out from behind the bushes, it was so random!
    The street cleaner never comes down our street. How random.
    
  • unexpectedly great.
    The party was totally random.
    
$\endgroup$
0
5
$\begingroup$

There are already too many great examples mentioned by Glen and Nick... not much left!

Some aspects of regression

  • error term and residual (it is somewhat funny when people are proud their residuals are uncorrelated with the regressors)

  • prediction and estimation (should we even stop making the distinction when they are about the predicted random effects?)

  • prediction/forecast interval versus confidence interval. I think there is a probability > 0.5 to quote the wrong one.

  • regressor (column in the design matrix) versus covariable et al. Especially in technical situations where the distinction is essential, many people (including myself) tend to be imprecise.

$\endgroup$
1
  • $\begingroup$ Sorry I'm confused. Is there a difference between prediction and estimation? Could you explain more on your last two points as well? Thanks! $\endgroup$
    – yuqian
    Mar 30, 2016 at 2:22
5
$\begingroup$

In insurance environments especially, it is common to use variance to refer to any sort of difference, rather than the mean of the squared differences between each data point and the mean of the data set.

$\endgroup$
1
  • 7
    $\begingroup$ I too have met variance used in this different sense, but note that variance was an existing English word when R.A. Fisher hijacked it for this purpose in 1918. So this is a different use; statistical people can't claim ownership of the true meaning. $\endgroup$
    – Nick Cox
    Mar 23, 2016 at 12:40
5
$\begingroup$

Concerning regression models, many newbies confuse multiple with multivariate regression. It has been stated elsewhere on this site and it may be worth reiterating here as well:

Multiple regression refers to a regression model with more than one covariate (aka independent variable) and a single response (aka dependent) variable.

Multivariate regression refers to a regression model dealing with a vector-valued response.

NaN is different from a missing value, commonly denoted by NA. A NaN is typically delivered by a statistical algorithm that fails to compute a numerical value as expected (e.g. when a division by 0 or the log of a negative number is attempted). An NA happens when, for some reason, the requested information is not available (e.g. in questionnaire data a person refuses to provide his/her income.)

A dataset with NAs is not equivalent to the same dataset with NAs replaced by some arbitrary number such as -999 or -9999.

$\endgroup$
4
$\begingroup$

One of the most harmful misnomers is "type I error rate". $\alpha$ is not a rate (rates can be larger than 1.0) and it is not the probability of being in error. It is best called type I assertion probability to make it clear how limited the concept actual is. More here.

$\endgroup$
3
  • 1
    $\begingroup$ +1. I was waiting, Frank, for this post, as it has been countless of times you have made aware of the historical misuse of the terms here. I hope the mainstream books soon adopt the more apt terminology. $\endgroup$ Aug 1, 2023 at 15:02
  • 1
    $\begingroup$ $x$ can’t be greater than 1 doesn’t mean it can’t be a rate. I’m not a native English speaker but maybe that this explains why Fisher says it’s not a rate: "A rate is a special ratio in which the two terms are in different units. For example, if a 12-ounce can of corn costs 69¢, the rate is 69¢ for 12 ounces. This is not a ratio of two like units, such as shirts. This is a ratio of two unlike units: cents and ounces." $\endgroup$ Aug 1, 2023 at 15:21
  • $\begingroup$ Well said. I've seen a few people include rate as a term for probability. But more generally a rate is akin to a first derivative or to the way Fisher defined it. Thanks. $\endgroup$ Aug 1, 2023 at 15:27
3
$\begingroup$

Risk

Risk does not mean probability

Risk is the sum of the costs of all outcomes, each of these costs multiplied by the probability of them happening.

Risk is usually weighed against reward which is the gain that we are seeking to achieve.

Here is one example: How Deadly Is Your Kilowatt. Here the risks — number of dead people for different sources of energy — are weighed against the reward — terawatt hours of energy produced by these sources of energy.

So for instance: the risk of nuclear power is not the probability that a meltdown will happen; it is the probability that a meltdown will happen, multiplied by the number of people that die from it, summed with with the number of people that die from normal operations multiplied by the probability that operations remain normal.

$\endgroup$
4
  • 4
    $\begingroup$ "Risk" doesn't have a universally accepted standard definition. But, "the sum of the costs [losses] of all outcomes, each of these costs [losses] multiplied by the probability of them happening" is the definition of expected cost [loss]. Risk, on the other hand, generally refers to (adverse) deviations from the expected loss. So, your definition is expectation, while I think typical definitions of risk deal with dispersion. $\endgroup$
    – A. Webb
    Mar 23, 2016 at 14:27
  • $\begingroup$ For example, when we buy insurance, the purpose is to reduce risk (reduce the impact of unlikely events), but the actual expected costs are higher for the insured, the difference being the expenses and profits of the insurer. The extreme losses in the tail have been traded for a more steady cost of the premium. $\endgroup$
    – A. Webb
    Mar 23, 2016 at 14:29
  • 3
    $\begingroup$ @A.Webb FWIW, the (international) Society for Risk Analysis defines risk as "The potential for realization of unwanted, adverse consequences to human life, health, property, or the environment; estimation of risk is usually based on the expected value of the conditional probability of the event occurring times the consequence of the event given that it has occurred." Thus risk does appear to have a standard definition--and it shows you are right to distinguish risk from how it might be estimated or measured. $\endgroup$
    – whuber
    Mar 23, 2016 at 21:18
  • 2
    $\begingroup$ Risk, as used by epidemiologists, also means the rate at which the probability of an event occurs, or $P(A)/t$. $\endgroup$
    – Alexis
    Mar 23, 2016 at 22:43
3
$\begingroup$

Bayesian

Students learning it might not have trouble telling you whether something "looks" Bayesian, but ask them to solve a problem with a frequentist and a Bayesian approach and they'll probably fail.

In my experience students end up being taught that it's just a philosophical difference, with no concrete example that shows the same problem being attacked with both approaches.

Now ask them why someone might take a frequentist approach in their example; chances are their best explanation would be something like "well, back in the old days, computers didn't exist..."

$\endgroup$
2
  • 1
    $\begingroup$ Could you share your explanation why someone might take a frequentist approach? Thanks! $\endgroup$
    – yuqian
    Mar 30, 2016 at 2:20
  • 4
    $\begingroup$ @yuqian: Yeah. For me, the important part is that you do it when you want people to objectively agree with you. Bayesian approaches require prior distributions, which are inherently subjective, and in real-world problems there is no single objectively-correct prior... which means two people can compute different answers for the same problem depending on what they think their priors should be. With a frequentist approach there is no such ambiguity, and that makes it possible to compare your results with those of others in an objective way. $\endgroup$
    – user541686
    Mar 30, 2016 at 3:02
2
$\begingroup$

Statistical terms probably should not be corrected if they are not being used in a specifically statistical context and the terms have colloquial meanings that are consistent with the message communicated.

For instance in the case of the Al Gore plot of temperatures and atmospheric CO2 content, "correlation" can mean simply "a mutual relationship or connection between two or more things.", which is exactly what Gore was suggesting, there are mutual relationships between global temperatures and atmospheric CO2 (the causality is in both directions). So it would be at best pedantry to substitute the statistical definition of correlation where there is no strong reason to think that was what Gore actually intended.

To take it further, in "correlation does not imply causation", "imply" has colloquial meanings which include "indicate the truth or existence of (something) by suggestion rather than explicit reference.", so outside statistics, correlation may well imply causation, depending on what was meant by "imply". Clearly even within statistics, correlation is evidence for causation. Indeed we can only sense correlation (things happening together) not causation (Hume) so correlation (in a colloquial sense) is usually how we arrive at causation.

“Correlation doesn’t imply causation, but it does waggle its eyebrows suggestively and gesture furtively while mouthing ‘look over there.’” Randal Munroe

So if we are going to correct misuse of statistical terms, we need to make sure first that they are being used in a specifically statistical context, and even then we should be trying to understand the substance of the message before worrying about the wording.

$\endgroup$
-1
$\begingroup$

Statistical significance & p-values

According to me, the most harmful words in the history of statistics are p-value and statistical significance. They should not only be corrected, but even deleted. The journal of Basic and Applied Social Psychology banned them (Trafimow & Marks, 2015). "Statistical significance" doesn’t mean anything, for any definition that I have heard, even with definitions that didn’t include p-values. The only similar expressions that may be meaningful are "clinically significant", "mechanically significant", "biologically significant", "YOUR_APPLICATION significant", etc.

Statistics are just numbers. A number becomes significant in light of an application IRL, it has no meaning by itself. So statistics, as a field, doesn’t provide significance; it provides numbers. If you obtain such and such statistic, it means that basing a decision on it, following a particular logic, will have a particular impact IRL. The significance of your statistic-based decision, and thus of the statistical value of interest, is the impact of that decision IRL. Thresholds 0.05, 0.01, 0.001, 0.0001, etc. are completely arbitrary and do not come from an impact analysis on real life. Impact analyses start with effect sizes ("The drug has this effect, which reduces symptoms by P%, which would have XYZ impact on patients’ health, etc.") or predictive values, for instance.

A flawed framework

Moreover, p-values and statistical significance are based upon a flawed framework. The null hypothesis significance testing (NHST) procedure has been thoroughly criticized through decades (see The ASA's Statement on p-Values: Context, Process, and Purpose and references). It’s not about Bayesians vs. frequentists, it’s just flawed. No Bayesian alternatives that I read convinced me, despite the fact that attaching probabilities to hypotheses can be meaningful in some contexts (e.g., when supposing that the drug that you’re studying is ineffective on some patients, or has a different effect).

Fixing things

Descriptive statistics and performance metrics are enough to evaluate the quality of a model/hypothesis. If you want to test the generalizability of the model/hypothesis, you need to repeat the experiment on a new population of interest. Studies after studies, experiments after experiments, you’ll likely find better estimates of the reality. This is systematic empiricism, or science in other words.

For example, systematic empiricism is how our predecessors improved their understanding of gravity. The first humans proposed the simple model "object falls to the ground". Then, they started building catapults and formalized more and more the process of falling, taking into account the mass of the object, the distance, the time it would take to fall, etc. Centuries after, Newton did precise tests using more elaborate mathematical tools and created a model including Newton’s laws of motion. But Einstein wasn’t satisfied with it, so he proposed the theory of general relativity, etc.

These theories, which have been being more and more precise, have never needed a p-value or "statistical significance" to be validated. You simply need to measure where objects fall, admit it’s the truth, and then build a model that predicts it. The challenge is to build a model that predicts well in different context. In any case, any fall you observe remains the truth. If you change the angle of your throw, it will fall somewhere else. You don’t need to run a "significance test" on a sample of falls to know whether the objects have fallen somewhere else, you’ve just observed where they have fallen. The only statistics you need are those who describe the falls (where, when, with what angle, etc.) and the difference between your prediction and the reality, whereupon you build a model that can reduce this difference. P-values and "significance tests" don’t tell you where object falls (what is the reality), they don’t tell you if your predictions were right, and they don’t tell you how to reduce the difference between the prediction and the reality. So ban them.

$\endgroup$
2
  • 3
    $\begingroup$ I don't think this addresses the question, which is about terminology that is used incorrectly. $\endgroup$
    – mkt
    Aug 1, 2023 at 14:33
  • $\begingroup$ @mkt "Equally spread is the use of correlation instead of association. Or correlation implying causation." ==> Equally spread is the use of "statistical significance" instead of nothing or "clinically significant". Or "low p-values imply no effect", etc. That’s why I commented. The question also addresses the interpretation of terms. $\endgroup$ Aug 1, 2023 at 15:03

Not the answer you're looking for? Browse other questions tagged or ask your own question.