Estimate Box-Cox Transformation Lambda Using Skewness and Kurtosis

Question

I would be interested in a method to find an appropriate Lambda parameter for the Box-Cox transformation based on only the skewness and the kurtosis of a given sample.

I.e, if the skewness and kurtosis indicate that the data might be normal (i.e., skewness of 0 and kurtosis of 3) the method should choose a Lambda of 1. If the skewness and kurtosis deviate from these values, the Lambda should be chosen such that the deviation are corrected as well as possible.

I was not able to find prior work that proposes such a method. Is there someone who might be able to help me out in this regard?

Answer 1

There are several reasons why you have not found any such procedure. Some are very simple and may seem utterly obvious, but that doesn't make them less compelling.

1. Skewness and kurtosis in the sense of moment-based measures are just one possible choice of measures of "skewness" or "kurtosis" in some Platonic sense. To be less cryptic, there are vague concepts of asymmetry and tail weight of distributions that can be made precise and quantified in many different ways. Moment-based measures can be, for example, over-sensitive to outliers and they pose more subtle problems in being limited as functions of sample size, so that a sample may not be able to exhibit the skewness and kurtosis of its parent family.

2. Skewness and kurtosis can be calculated for counted or measured variables regardless of whether values are negative, zero or positive, but the Box-Cox power family in its simplest form usually presupposes positive or at least non-negative values.

3. Similarly skewness and kurtosis can be calculated for bounded variables -- such as those observed on, or mappable to, $(0, 1)$ or $[0, 1]$ -- but if transformations make sense for the latter they are usually of different form, e.g. logit rather than logarithm. An even more extreme case is that of $(0, 1)$ indicator variables which often exhibit extreme skewness but which cannot be transformed in any useful way.

4. Such a formulation places even more stress in the wrong place than do naive or over-simplified treatments of Box-Cox by arguing or implying that getting closer to normal marginal distributions is the main deal. On the contrary, even for regression-type models it's at most conditional distributions being close to normal that is some kind of ideal. More generally, what is most valuable about a transformation -- if it is valuable at all -- is likely to be getting closer to (in rough order of importance) additivity, linearity, homoscedasticity and symmetry -- with normality as a special case of the latter being least important of all ideal conditions (often unfortunately stated as assumptions).

5. Transformations divide the statistical world, and this community too. There are leading members here who never saw a transformation they didn't dislike -- and leading members too who are very positive about transformations being often useful when carefully chosen (above all for visualizing data or results). But there is perhaps slowly and steadily growing recognition that (in jargon introduced with generalized linear models) using link functions other than identity is in many ways a deeper and more helpful way to deal with awkward (e.g. skewed or long-tailed) outcomes. A canonical example here is Poisson regression which does not entail transforming an outcome but respects the scale of the outcome by estimating positive mean functions. Transformations can still make sense for predictors.

6. I take from the Box and Cox analysis two main ideas, neither quite original even at the time, but both pushed well in their paper: (a) the most common transformations aren't a ragbag of small mathematical tricks but form a family (b) the data themselves can indicate a transformation that may be appropriate, or (just as valuable) that a range of transformations may be appropriate. It's notable that their empirical examples are used to motivate logarithmic and reciprocal transformations that would have seemed sensible to experienced practitioners any way. What I think is contrary to the spirit of their paper is any idea that you can and should automate choice of transformation or that there are calculations that will indicate that power 0.123 or whatever is the transformation to use. Their intent aside, in 60 years since their paper (1964), other methods have emerged strongly that offer other solutions in different ways given awkward distributions.

7. When faced with the same variables in different groups or in different datasets, it is usually more valuable to treat them consistently than to optimize in terms of what seems best to suit particular skewness or kurtosis. Typically a scale -- in terms of either a transformation or a link function -- should be chosen consistently on substantive or scientific grounds (e.g. physical, biological, economic). Most often that means working on logarithmic scale.

In short, this procedure doesn't obviously exist -- and isn't even a good idea in principle -- because you need to know much more than skewness and kurtosis to know whether a transformation is a good idea at all, let alone which transformation(s) are worth trying, principally

• what model is envisaged

• what variables are candidates for inclusion in a model

• what role does any variable play, outcome or predictor

• what is the range of values observed in practice

• what is the range of values possible in principle

Answer 2

I'm more of someone @nick-cox terms "I never met a transformation I didn't dislike" kind of statistician. It is ironic that we seek transformations that fit the data, yet "fit the data" means agreement with the empirical cumulative distribution function (ECDF). ECDF-guided model selection will inherit the variance of ECDF so provides little advantage over just using the ECDF. This is what ordinal semiparametric regression models do, as described here. You do not need to worry about the transformation of $Y$. Ordinal regression takes into account transformation uncertainty, unlike trial-and-error procedures which result in too-narrow confidence bands by not taking model uncertainty into account. These are reasons the Cox proportional hazards model is so popular, and why the proportional odds model is rapidly gaining in popularity.

In addition to handling extreme skewness and kurtosis, ordinal models handle bimodality, detection limits, and floor and ceiling effects.

Answer 3

The answer by @Nick Cox is very thorough and informative and I believe you should accept it.

Although Nick is correct that such a procedure does not exist, I wanted to provide you with some tools that may be at least partially useful for you. The short of it is, you can try to find a distribution (or several) which is consistent with the data you have (even if that is limited the the skew and kurtosis). After you have identified a reasonable distribution, you may be able to find a custom transformation that works for you. For example, if an Exponential distribution seems to provide a reasonable fit for $X$, then the transformation $U = -\log X$ will be approximately uniform on $(0, 1)$, which can be easily transformed to a logistic or Gaussian distribution (if you really really need this).

---

Cullen-Frey Plot

The R package fitdistrplus provides a nice implementation of the Cullen-Frey plot, which attempts to identify a distribution based only on the kurtosis and the square of the skewness.

# Simulate some data
n <- 500
y <- rgamma(n, 3, 1.5) + rlnorm(n, 1, 0.5)

#install.packages("fitdistrplus")
library(fitdistrplus)
descdist(y)

This figure suggests that a lognormal distribution may be roughly consistent with these data. This in-turn suggests that the transformation $\log X$ will yield data which may be approximately Gaussian.

---

The Normal-Wald Triangle Plot

The Normal-Wald distribution is a highly flexible 3-parameter distribution which can capture both left and right skew as well as a wide variety of tail behaviors.

The Normal-Wald triangle plot is a useful tool for assessing the behavior of a data-set, using steepness (akin to kurtosis) and asymmetry (akin to skewness). See this paper or Sections 3.4.1 and SM6 of this paper for details.

library(devtools)
#devtools::install_github("knrumsey/GBASS")
library(GBASS)
X <- matrix(rep(1, n), ncol=1)
fit <- nwbass2(X, y,
               m_gamma=-1.4, s_gamma=82,
               m_beta=0, s_beta=0.5)
nw_triangle(fit, bg='orange', pch=21, cex=1.5, details=TRUE)

# Alternatively, gain point estimates only using the moments
# Fix location = 0, scale = 1
skew <- mean(((y-mean(y))/sd(y))^3) # sample skewness
kurt <- mean(((y-mean(y))/sd(y))^4) # sample kurtosis
stats <- c(NA, NA, skew, kurt)
nw_est_mom(stats=stats, mu=0, delta=1, triangle=TRUE)
## returns: (0.3089, 0.6740)

# With an additional moment, we can estimate the scale param as well
stats <- c(NA, var(y), skew, kurt)
nw_est_mom(stats=stats, mu=0, triangle=TRUE)
## returns: (0.4437, 0.7399)

As with Cullen-Frey, this figure indicates that a log-normal distribution may be consistent with these data.

Answer 4

Unless you impose some pretty restrictive conditions, using skewness and kurtosis as a basis to choose a transformation is not useful.

Indeed a number of textbooks used in some particular application areas contain prescriptions about how much skewness suggest either an identity, a square root or a log transformation (some may suggest some other transformations in addition or in place of some of these). As we will see below, such advice based on standardized moments is clearly misplaced in general; indeed in some cases such advice may well make things worse.

There's multiple ways to see that distributions with the same skewness (or even skewness and kurtosis together) may need very different transformations. Consequently skewness and kurtosis cannot of themselves ever hope to be sufficient as a way to identify a suitable power transformation.

1. A location-shift will leave skewness and kurtosis unchanged but (assuming it remains within the region where such transformations work) will certainly change the impact of the transformation. e.g. if $X_1 \sim \text{Exp}(1)$ and $X_2=X_1+5$, then $\text{skew}(X_1) = \text{skew}(X_2)$ and $\text{kurt}(X_1)=\text{kurt}(X_2)$ but a transformation that works quite well for $X_1$ will not work at all for $X_2$.

   In the case of the exponential, $X_1^{p}$ where $p≈1/3.6$ works quite well as a symmetrizing transform (it's not very far from a monotonic transformation to exact normality and the choice of values near to the reciprocal of $3.6$ can be guessed from the connection to the Weibull), but $X_2^{p}$ is almost useless, in spite of their identical skewness and kurtosis.

   

   Here y corresponds to $X_1$ above and y1 corresponds to $X_2$.

   To be clear, I am not proposing that anyone should be actually shifting the origin of their data as I did here. The aim is just to show by the simplest means that we can have two distinct variables with identical skewness and kurtosis but very different response to transformation. Indeed in this example, all the higher standardized moments would be equal as well, so choosing still more such standardized-moment-based coefficients will not help; indeed anything that simply considers shape alone cannot suffice, even if you perfectly describe the shape, since the exponential and the shifted exponential have identical shape.

2. At the same time, exactly the same transformation may be suitable for a wide range of different values of skewness and kurtosis. For example, every lognormal distribution has the same symmetrizing transformation (the log, naturally), but skewness and kurtosis are both functions of the shape parameter $\sigma$. Indeed the skewness of a lognormal can be anything from as near to $0$ as you wish to extremely skewed (albeit still finite). Similarly the kurtosis can be anything from very close to that for a normal to extremely large. But the same transformation (the log) is the correct choice for all of them. So again, we are left to conclude that the skewness and kurtosis cannot necessarily help us choose the correct transformation in general.

   The lognormal is not the only distribution family with this property, it's just a simple example where the 'correct' transformation to use is not a matter for debate.

3. It's perfectly possible to construct pairs of distributions where power transformations make sense for both but where we have a pair of distributions where the weaker transformation goes with the more skewed and more kurtotic distribution.

   An example is provided by lognormal and gamma distributions. As we saw in point 2, the lognormal may be anywhere from quite near to the normal (only very mildly right skew and only very mildly more kurtotic than the normal) to very skewed and kurtotic, but we can choose a gamma distribution that's more skew and with larger kurtosis.

   Except for small values of the gamma shape parameter, the best choice of Box-Cox transformation will be close to the cube root (the Wilson-Hilferty transformation). For smaller values, a slightly stronger power transformation may do better (e.g. as we saw above, when the shape is $1$, the $3.6$th root is a good choice and for slightly smaller shape parameters the fourth root may be a reasonable choice), but for each of them the log transformation will make it left-skewed. The log is "too strong" for the gamma even when it's quite skewed.

   

   Here we have a lognormal on the left and a gamma on the right. The gamma is more skewed and kurtotic but the correct transformation for the lognormal would be too much for the gamma. As we see, the more skewed gamma only requires a cube root transformation.

   Here the $\sigma$ parameter of the lognormal was $0.1$, while the shape parameter of the gamma was $3$, resulting in greater skewness and kurtosis for the gamma, but nevertheless the gamma required a weaker transformation to get close to normality.

   In another of my answers (which I could not locate just now) I looked at a distribution family for which the square root was a good choice of power transformation. Nevertheless, it would be possible to choose a member of such a family that was more skewed than some gamma and which was in turn more skewed than some lognormal, where the the weaker power transformations would go with the stronger skewness and the strongest transformation would go with the weaker skewness.

4. We can readily find population distributions which are quite distinctly non-normal but which have the same skewness and kurtosis as the normal (there are both discrete and continuous examples as well as examples which are neither). Naturally a skewness-and-kurtosis-based rule looking at the whole distribution (or even very large samples) would not transform these distributions at all, leaving us with the distinctly-non-normal distributions we began with. We can widen the class of allowed transformations and (assuming our example was one of the continuous possibilities) identify exact transformations to normality, but these perfect transformations could not be indicated by the skewness and kurtosis.

Many other examples may be produced which clearly show that skewness and kurtosis are not of themselves a good guide to choice of transformation.