What other normalizing transformations are commonly used beyond the common ones like square root, log, etc.?

Question

In the analysis of test scores (e.g., in Education or Psychology), common analysis techniques often assume that data are normally distributed. However, perhaps more often than not, scores tend to deviate sometimes wildly from normal.

I am familiar with some basic normalizing transformations, like: square roots, logarithms, reciprocal transformations for reducing positive skew, reflected versions of the above for reducing negative skew, squaring for leptokurtic distributions. I have heard of arcsine transformations and power transformations, though I am not really knowledgeable about them.

So, I am curious as to what other transformations are commonly used by analysts?

Answer 1

The Box-Cox transformation includes many of the ones you cited. See this answer for some details:

• How should I transform non-negative data including zeros?

UPDATE: These slides provide a pretty good overview of Box-Cox transformations.

Answer 2

The first step should be to ask why your variables are non-normally distributed. This can be illuminating. Common findings from my experience:

• Ability tests (e.g., exams, intelligence tests, admission tests) tend to be negatively skewed when there are ceiling effects and positively skewed when there are floor effects. Both findings suggest that the difficulty level of the test is not optimised for the sample, either being too easy or too difficult to optimally differentiate ability. It also implies that the latent variable of interest could still be normally distributed, but that the structure of the test is inducing a skew in the measured variable.
• Ability tests often have outliers in terms of low scorers. In short there are many ways to do poorly on a test. In particular this can sometimes be seen on exams where there are a small percentage of students where some combination of lack of aptitude and lack of effort have combined to create very low test scores. This implies that the latent variable of interest probably has a few outliers.
• In relation to self-report tests (e.g., personality, attitude tests, etc.) skew often occurs when the sample is inherently high on the scale (e.g., distributions of life satisfaction are negatively skewed because most people are satisfied) or when the scale has been optimised for a sample different to the one the test is being applied to (e.g., applying a clinical measure of depression to a non-clinical sample).

This first step may suggest design modifications to the test. If you are aware of these issues ahead of time, you can even design your test to avoid them, if you see them as problematic.

The second step is to decide what to do in the situation where you have non-normal data. Note transformations are but one possible strategy. I'd reiterate the general advice from a previous answer regarding non-normality:

• Many procedures that assume normality of residuals are robust to modest violations of normality of residuals
• Bootstrapping is generally a good strategy
• Transformations are another good strategy. Note that from my experience the kinds of mild skew that commonly occur with ability and self-report psychological tests can usually be fairly readily transformed to a distribution approximating normality using a log, sqrt, or inverse transformation (or the reversed equivalent).

Answer 3

John Tukey systematically discusses transformations in his book on EDA. In addition to the Box-Cox family (affinely scaled power transformations) he defines a family of "folded" transformations for proportions (essentially powers of x/(1-x)) and "started" counts (adding a positive offset to counted data before transforming them). The folded transformations, which essentially generalize the logit, are especially useful for test scores.

In a completely different vein, Johnson & Kotz in their books on distributions offer many transformations intended to convert test statistics to approximate normality (or to some other target distribution), such as the cube-root transformation for chi-square. This material is a great source of ideas for useful transformations when you anticipate your data will follow some specific distribution.

Answer 4

For skewed and heavy tailed data I use (and developed) the Lambert W x F distribution framework. Skewed and heavy-tailed Lambert W x F distributions are based on a non-linear transform of an input random variable (RV) $X \sim F$ to output $Y ~ Lambert W \times F$, which is similar to X but skewed and/or heavy tailed (see papers for detailed formulas).

This works in general for any continuous RV, but in practice we are mostly interested in Gaussian $X \sim N(\mu, \sigma^2)$. For heavy-tailed Lambert W x F distributions the inverse is bijective and can be estimated from the data using your favorite estimator for the parameter $\theta = (\mu_x, \sigma_x, \delta, \alpha)$ (MLE, methods of moments, Bayesian analysis, ...). For $\alpha \equiv 1$ and X being Gaussian it reduces to Tukey's h distribution.

Now as a data transformation this becomes interesting as the transformation is bijective (almost bijective for skewed case) and can be obtained explicitly using Lambert's W function (hence the name Lambert W x F). This means we can remove skewness from data and also remove heavy tails (bijectively!).

You can try it out using the LambertW R package, with the manual showing many examples of how to use it.

For applications see these posts

• What's the distribution of these data?: this has a full illustration of how to transform data to normality in R using the LambertW package.
• Looking for a distribution where: Mean=0, variance is variable, Skew=0 and kurtosis is variable
-

Answer 5

A simple option is to use sums of scores instead of the scores themselves. The sum of distributions tends to normality. For example, in Education you could add a student's scores over a series of tests.

Another option, of course, is to use techniques that do not assume normality, which are underestimated and underused.