Is there such a thing as a trimmed skewness estimator?

Question

I was wondering whether there is some equivalent of trimmed mean when estimating skewness (or kurtosis)? I am looking for something which would be called maybe a trimmed sample skewness or kurtosis.

If I had reason to believe that the outliers are generated by a different process than the rest of the data, I could just throw away the largest 10% and smallest 10% of observations (or other arbitrary %) and calculate the sample skewness only with the rest.

So the formula would then look like (for a series with mean zero):

$$\text{skewness} = \sqrt N \sum_{i=1}^n \left( \left (x_i : q_.1(x) < x_i < q_.9(x))^3 \right)/( \text{Var}(x)^{\frac{3}{2}})\right)$$

Where $q_{.1}$ is the 10-th percentile of the $x$ distribution and $q_{.9}$ is the 90-th percentile.

I suppose the $\text{var}(x)$ would have to be adjusted in a similar way (i.e. some part of x values also thrown away). But my question is: is there such a kind of measure, is it used? if so, where could i find its properties? If it's not used, is there some reason why (maybe because it doesn't make any sense whatsoever)?

I am aware there are some other robust estimators of skewness and kurtosis, such as those discussed in this paper. I am also aware that saying "just trim" is probably wrong and there should be some assumptions behind throwing away some subset of my data.

Answer 1

I see no problem in principle behind trimmed skewness or trimmed kurtosis. You don't need a formula; you just need a recipe, say

1. Trim $p$% in one tail and $q$% in another tail. (Commonly $p = q$ but not necessarily.)

2. Calculate skewness and kurtosis for what remains.

I would counsel strongly against wording such as "throwing away" or "throwing out". I have never encountered a situation in which anyone knows exactly what $p$ or $q$ should be. Better practice is to trim, ideally with some sensitivity analysis, using at least two different amounts of trimming, certainly also $p = q = 0$.

So, I can imagine saying the skewness is such-and-such, but that's mostly a side-effect of a few outliers, as shown by the fact that the 5% trimmed skewness is such-and-such. Some adhockery of use seems inescapable but defensible with some sensitivity analysis or cautious wording.

A loosely related idea is that skewness is well thought of as a function calculated from sample data, most simply as the means of order statistics with the same depth, as a function of depth or min(plotting position, 1 $-$ plotting position).

So order a sample of size $n$ as $x_{(1)} \le x_{(2)} \le \cdots \le x_{(i)} \le \cdots \le x_{(n-1)} \le x_{(n)}$. Then in a symmetric distribution the means

$$(x_{(1)} + x_{(n)})/2, (x_{(2)} + x_{(n-1)})/2, \cdots$$

are expected to be equal, while in a skewed distribution the means will vary, yet the pattern of the means will show how far this is strongest for outliers. The depths of these means are respectively $1, 2, \cdots$ and the plotting positions $(i - 0.5)/n$ or whatever variant appeals. See this thread, including several of the comments.

I don't think that trimmed skewness or kurtosis is very much used in practice, partly because

1. If the skewness and kurtosis are highly dependent on outliers, they are not necessarily useful measures, and trimming arbitrarily solves that problem by ignoring it.

2. Problems with inconvenient distribution shapes are often best solved by working on a transformed scale.

3. There can be better ways of measuring or more generally assessing skewness and kurtosis, such as the method above or $L$-moments. As a skewness measure (mean $-$ median) / SD is easy to think about yet often neglected; it can be very useful, not least because it is bounded within $[-1, 1]$.

4. It is a complication you have to explain. Unusual measures have to get past reviewers, which is a brake on adoption of unusual measures.

Answer 2

If you really believe the outliers come from a different process, then why not model that with a mixture model? Trimming and Winsorizing are crude solutions to say the least.

Also, with regard to kurtosis, it is mainly determined by the tails: For all possible distributions of random variables $X$ having finite fourth moment, kurtosis is within $\pm 0.5$ of $E\{Z^4*I(|Z| > 1)\} + 0.5$, where $Z = (X - \mu)/\sigma$.

Further, if you restrict the class of distributions to continuous distributions where the density of $Z^2$ decreases on the $[0,1]$ interval, then kurtosis is within $\pm 0.25$ of $E\{Z^4*I(|Z| > 1)\} + 0.25$.

Finally, for any sequence of distributions where kurtosis $k$ tends to infinity, $\lim E\{Z^4*I(|Z| > b)\}/k \rightarrow 1$, for every real number $b$.

The previous three statements are mathematical theorems proven in my paper "Kurtosis as Peakedness, 1905 - 2014. R.I.P.," and together explain why kurtosis is a measure of the tails of the distribution, and not the peak or center.

Some have provided "counterexamples" to the fact that kurtosis measures tails, but these are red herrings: If you define a tail measure that is not kurtosis, then of course kurtosis does not measure that particular tail measure. There are infinitely many measures of tail.

On the other hand, there can be no counterexamples to the theorems I just stated.

All of which is to say that if you truncate the tails, you no longer have anything related to kurtosis. Actually, you would have something much closer to kurtosis if you reverse trim, replacing all $Z$ within $[-1,1]$ with 0, and then finding the average of the resulting $Z^4$ values. By the first theorem above, the actual kurtosis is between the result of that calculation, and the result of that calculation +1.

Finally, these theorems apply to any distributions, including empirical distributions. With empirical distributions, $Var(X)$ has $n$ in the denominator rather than $n-1$, so the $Z$-values must be defined using that variance estimate. The kurtosis of the empirical distribution is then the average of the resulting $Z$ values, each taken to the fourth power.