# What concepts/objects are "wrongly" formed in probability and statistics?

Some background: There is a wonderful mathematical article which argues that mathematicians have been wrong to frame mathematical formulae in terms of the constant $$\pi$$, and that they should have framed these things in terms of $$2 \pi$$ (the quantity for 'one turn', which should have its own symbol) instead:

Palais (2001) $$\pi$$ is wrong! The Mathematical Intelligencer 23(3), pp. 7-8

The rumblings from this article have led to a movement within the mathematical profession to remove references to $$\pi$$ in mathematical work and instead frame things in terms of the constant $$\tau = 2 \pi = 6.283185...$$ (see e.g., the Tau manifesto). There is quite a bit of argument to and fro as to which is the appropriate symbol for framing mathematical results, and whether or not $$\tau$$ should supplant $$\pi$$. My own view is that the malcontents make a compelling case, but that is not really at issue here.

My question: Are there any analogous cases in probability and statistics where a concept or object has been formulated in a way that, in hindsight, is not the best way to frame it? If so, what is the best way to frame the concept/object in question?

• Clearly there couldn't be a single canonical answer to this, so I've converted the post to CW. Commented Feb 10, 2021 at 1:39
• A sufficiently partisan Bayesian might answer "just about everything". Commented Feb 10, 2021 at 12:02
• Were it the case, mathematicians would have made another wrong choice given that $\tau$ is frequently used in physics to represent time constants, and frequently multiplied by $2\pi$. Commented Feb 10, 2021 at 17:26
• @MassimoOrtolano just use $\tau^2$, easy ;) Commented Feb 10, 2021 at 17:34
• @JohnColeman: Indeed! Nevertheless, what I am looking for here is cases where a statistical concept is wrong on its own terms --- i.e., with respect to the purpose it is created to perform.
– Ben
Commented Feb 10, 2021 at 21:20

## The chi-squared distribution is wrong!

The two main statistical uses of the chi-square distribution are to give the asymptotic (and in some cases exact) distribution of variance estimators, and to give the asymptotic distribution of test statistics relating to squared deviations. In both cases the chi-squared distribution is on the wrong "scale" and this means that results pertaining to the chi-squared distribution generally require the inclusion of a scaling constant to put the quantity of interest on the same scale as the distribution.

For example, consider some of the standard statistical results where we use the chi-squared distribution. When looking at the distribution of the sample variance $$S^2$$ or the true-mean-adjusted sample variance $$S_\mu^2$$, we have the standard asymptotic results (exact for normal data):

\begin{align} n \cdot \frac{S_\mu^2}{\sigma^2} &\sim \text{ChiSq}(n), \\[12pt] (n-1) \cdot \frac{S^2}{\sigma^2} &\sim \text{ChiSq}(n-1). \\[6pt] \end{align}

Similarly, when constructing the F distribution, for testing ratios of variances, we define the F-statistic as:

$$F = \frac{\chi_{n_1}^2/n_1}{\chi_{n_2}^2/n_2} \sim \text{F}(n_1,n_2).$$

We get a similar scaling issue when we use the Pearson test statistic. In this test the framing in terms of the chi-squared distribution works on the scale of expected cell counts rather than the more natural scale of probabilities. This is easily seen when examining the Pearson test for multinomial data. With $$n$$ values over $$k$$ categories we get the test statistic and null distribution:

$$n \sum_{i=1}^k \frac{(\hat{p}_i - p_i)^2}{p_i} \overset{H_0}{\sim} \text{ChiSq}(n-1).$$

Observe that in all these statistical results there is an annoying scaling constant, which is there solely to make up for the fact that the chi-squared distribution is on an unnatural scale to begin with (i.e., the chi-squared distribution is wrong!). A simpler framing of these results can be obtained by re-scaling the distribution so that it is on a more natural scale to begin with, which means that we no longer need scaling constants in the equations.

(An obvious corollary of my position is that the chi distribution is also wrong --- I recommend that this distribution also be reframed mutatis mutandis with the chi-squared distribution.)

Re-framing the distribution: To reframe these distributions, let's consider a set of IID standard normal random variables $$Z_1,...,Z_k \sim \text{N}(0,1)$$ and then denote the mean-of-squares of these values as:

$$\eta_k^2 = \frac{\chi_k^2}{k} = \frac{1}{k} \sum_{i=1}^k Z_i^2.$$

This leads us to define the eta-squared distribution, which is a scaled version of the chi-squared distribution, scaled to give unit mean. The distribution $$\eta_k^2 \sim \text{EtaSq}(k)$$ can be characterised by its density functions:

\begin{align} \text{EtaSq}(x|k) = \text{Ga}(x|\tfrac{k}{2},\tfrac{k}{2}) = \frac{(k/2)^{k/2}}{\Gamma(k/2)} \cdot x^{k/2-1} \exp \bigg( -\frac{k x}{2} \bigg) \cdot \mathbb{I}(x \geqslant 0), \end{align}

The eta-squared distribution has mean $$\mathbb{E}(\eta_k^2) = 1$$ and variance $$\mathbb{V}(\eta_k^2) = \tfrac{2}{k}$$, so it is trivial to see that $$\eta_k^2 \rightarrow 1$$ as $$k \rightarrow \infty$$. This distribution allows us to re-frame various standard results in probability and statistics into simpler forms.

Advantage 1: We now have simpler sample variance results. If we have $$n$$ data points then the sample variance distributions can be written as:

$$\frac{S^2}{\sigma^2} \sim \text{EtaSq}(n-1) \quad \quad \quad \frac{S_\mu^2}{\sigma^2} \sim \text{EtaSq}(n).$$

These are simpler equations than those framed in terms of the chi-squared distribution. As previously noted, it is trivial from the moments to see that this ratio converges to one as $$n \rightarrow \infty$$, which means that the sample variance converges to the true variance.

Advantage 2: We now have a more natural transition into the F distribution, which is just the distribution of the ratio of independent eta-squared random variables. Taking independent values $$\eta_{k_1}^2 \sim \text{EtaSq}(k_1)$$ and $$\eta_{k_2}^2 \sim \text{EtaSq}(k_2)$$ we have:

$$F = \frac{\eta_{k_1}^2}{\eta_{k_2}^2} \sim \text{F}(k_1,k_2)$$

Advantage 3: We now have the simpler version of the Pearson test for multinomial data. With $$n$$ values over $$k$$ categories we get the test statistic and null distribution:

$$\frac{n}{n-1} \sum_{i=1}^k \frac{(\hat{p}_i - p_i)^2}{p_i} \overset{H_0}{\sim} \text{EtaSq}(n-1).$$

Whilst there is still a scaling constant in this equation, for large $$n$$ this scaling constant barely affects the result, and we can get a reasonable result even if we remove it. In any case, the result is more natural than the corresponding result for the chi-squared distribution, insofar as we are now working on the scale of probabilities rather than expected counts.

• Excellent response, @Ben. This has always bothered me too! +10. Commented Feb 10, 2021 at 0:41