23
$\begingroup$

For features that are heavily skewed, the Transformation technique is useful to stabilize variance, make the data more normal distribution-like, improve the validity of measures of association.

I am really having trouble understanding the intuition behind Box-Cox transform. I mean how to configure data transform method for both square root and log transform and estimating lambda.

Could anyone explain in simple words (and maybe with an example) what the intuition behind Box-Cox transform is?

$\endgroup$
3
  • 1
    $\begingroup$ Could you tell us more on what is unclear for you? $\endgroup$
    – Tim
    May 20, 2020 at 7:47
  • $\begingroup$ On how box cox transformation works? $\endgroup$ May 20, 2020 at 7:49
  • 1
    $\begingroup$ See stats.stackexchange.com/questions/292417 for why $\lambda=0$ corresponds to the logarithm. $\endgroup$
    – whuber
    May 20, 2020 at 11:28

3 Answers 3

29
$\begingroup$

The design goals of the family of Box-Cox transformations of non-negative data were these:

  1. The formulas should be simple, straightforward, well understood, and easy to calculate.

  2. They should not change the middle of the data much, but affect the tails more.

  3. The family should be rich enough to induce large changes in the skewness of the data if necessary: this means it should be able to contract or extend one tail of the data while extending or contracting the other, by arbitrary amounts.

Let's consider the implications of each in turn.

1. Simplicity

Linear transformations--those of the form $x\to \alpha x + \beta$ for constants $\alpha$ and $\beta$--merely change the scale and location of data; they cannot change the shape of their distribution. The next simplest formula is to consider power transformations, of the form $x\to x^\lambda$ for (nonzero) constant $\lambda.$

2. Stability

A power transformation enjoys the nice property that rescaling the data results in rescaling their powers. That is, multiplying the data $x$ by some positive scale factor $\alpha$ results in multiplying $x^\lambda$ by $\alpha^\lambda.$ OK, it's not the same scale factor, but it is still just a rescaling.

In light of this, let's always standardize any batch of data $(x_1, x_2, \ldots, x_n)$ by rescaling it to place its center (perhaps its median) at $1.$ Specifically, this replaces each $x_i$ by $x_i$ divided by the middle value of all the $x$'s. This won't change the shape of the data distribution--it really amounts to choosing a suitable unit of measurement for expressing the values. For those who like formulas, let $\mu$ be the median of the batch. We will be studying the transformations

$$x \to \frac{(x/\mu)^\lambda - 1}{\lambda} = \frac{\mu^{-\lambda}}{\lambda}\,x^\lambda + \frac{-1}{\lambda} = \alpha\, x^\lambda + \beta$$

for various $\lambda.$ The effects of $\alpha$ and $\beta$ (which depend on $\lambda$ and $\mu$) on $x^\lambda$ do not change the shape of the distribution of the $x_i^\lambda.$ In this sense, the Box-Cox transformations of the standardized data really are just the power transformations.

Because we have made $1$ the central value of the batch, design crition 2--"stability"--requires that different values of the power $\lambda$ have relatively little effect on values near $1.$

Let's look at this in a little more detail by examining what a power does to numbers near $1.$ According to the Binomial Theorem, if we write $x$ as $x=1+\epsilon$ (for fairly small $\epsilon$), then approximately

$$(1 + \epsilon)^\lambda = 1 + \lambda \epsilon + \text{Something}\times \epsilon^2.$$

Ignoring $\epsilon^2$ as being truly tiny, this tells us that

Taking a power $\lambda$ of a number $x$ near $1$ is a nearly linear function that changes the distance between $x$ and $1$ by a factor $\lambda.$

In light of this, we can match the effects of different possible $\lambda$ by means of a compensating division of the distance by $\lambda.$ That is, we will use

$$\operatorname{BC}_\lambda(x) = \frac{x^\lambda - 1^\lambda}{\lambda} = \frac{x^\lambda - 1}{\lambda}.$$

The numerator is the (signed) distance between the power transform of $x$ and the power transform of the middle of the data ($1$); the denominator adjusts for the expansion of $x-1$ by the factor $\lambda$ when taking the power. $\operatorname{BC}_\lambda$ is the Box-Cox transformation with parameter $\lambda.$

By means of this construction, we guarantee that when $x$ is close to a typical value of its batch of data, $\operatorname{BC}_\lambda(x)$ will approximately be the same value (and close to zero) no matter what $\lambda$ might be (within reason, of course: extreme values of $\lambda$ can do extreme things).

3. Flexibility

We have many possible values of $\lambda$ to choose from. How do they differ?

This can be explored by graphing the Box-Cox transformations for various $\lambda.$ Here is a set of graphs for $\lambda \in \{-1,-1/2, 0, 1/2, 1, 2\}.$ (For the meaning of $\lambda=0,$ see Natural Log Approximation elsewhere on this site.)

Figure 1

The solid black line graphs the Box-Cox transformation for $\lambda=1,$ which is just $x\to x-1.$ It merely shifts the center of the batch to $0$ (as do all the Box-Cox transformations). The upward curving pink graph is for $\lambda=2.$ The downward curving graphs show, in order of increasing curvature, the smaller values of $\lambda$ down to $-1.$

The differing amounts and directions of curvature provide the desired flexibility to change the shape of a batch of data.

For instance, the upward curving graph for $\lambda=2$ exemplifies the effect of all Box-Cox transformations with $\lambda$ exceeding $1:$ values of $x$ above $1$ (that is, greater than the middle of the batch, and therefore out in its upper tail) are pulled further and further away from the new middle (at $0$). Values of $x$ below $1$ (less than the middle of the batch, and therefore out in its lower tail) are pushed closer to the new middle. This "skews" the data to the right, or high values (rather strongly, even for $\lambda=2$).

The downward curving graphs, for $\lambda \lt 1,$ have the opposite effect: they push the higher values in the batch towards the new middle and pull the lower values away from the new middle. This skews the data to the left (or lower values).

The coincidence of all the graphs near the point $(1,0)$ is a result of the previous standardizations: it constitutes visual verification that choice of $\lambda$ makes little difference for values near the middle of the batch.

Finally, let's look at what different Box-Cox transformations do to a small batch of data.

Figure 2

Transformed values are indicated by the horizontal positions. (The original data look just like the black dots, shown at $\lambda=1,$ but are located $+1$ units to the right.) The colors correspond to the ones used in the first figure. The underlying gray lines show what happens to the transformed values when $\lambda$ is smoothly varied from $-1$ to $+2.$ It's another way of appreciating the effects of these transformations in the tails of the data. (It also shows why the value of $\lambda=0$ makes sense: it corresponds to taking values of $\lambda$ arbitrarily close to $0.$)

$\endgroup$
9
$\begingroup$

Adding something to the great answer by whuber. Let's say you have $k$ independent random variables $X_1, X_2,..., X_k$ normally distributed with mean $m_i$ and variance $\sigma_i^2$ for $i=1,...,k$.

Now, let's assume that $\sigma_i = f(m_i)$ and $f$ is some known function. In simple situations we can guess this function, for example from a graph of sample standard deviation and sample mean. We want to find such a transformation $t$ that a sequence of independent random variables $Y_1 = t(X_1),...,Y_k = t(X_k)$ has (at least approximately) constant variance $\mathrm{Var}(Y_i) = const$ for $i=1,...,k.$

You can use Taylor expansion around mean to achieve this as follows

$$Y_i = t(X_i) \approx t(m_i)+t'(m_i)(X_i-m_i).$$

The condition of constant variance leads to differential equation $t'(x)f(x)=c$ and the transformation $t$ has the form $$t(x)=c_1 \int \frac{1}{f(x)}dx + c_2,$$

where $c_1$ and $c_2$ are constants. Note that if $f(x)=x$, then the transformation is $t(x)=\ln(x).$ If $f(x) = x^\alpha$ ($\alpha \neq 1$), then the transformation is $t(x) = \frac{1}{1-\alpha}x^{1-\alpha}.$ Using the well known fact that $\lim_{x\to 0} \frac{a^x-1}{x} = \ln(a)$ we finally get

$$t_\lambda(x) = \begin{cases} \frac{x^{\lambda}-1}{\lambda} & \lambda \neq 0 \\ \ln(x), & \lambda = 0 \end{cases} $$

for $x>0$, which is Box-Cox family of transformations. Transformation $t_\lambda(x)$ corresponds to $f(x) = x^{1-\lambda}.$

$\endgroup$
3
  • 4
    $\begingroup$ +1. But Normality is not a requirement for this analysis--which is a good thing, because typically these transformations are applied when the data grossly depart from Normality! See stats.stackexchange.com/a/251661/919 for instance. What's really good about this answer is that it addresses the question of transforming to stabilize variance, which is completely different than the question of transforming to change distributional skewness, which is motivated by a desire to achieve approximate symmetry of distribution. $\endgroup$
    – whuber
    May 20, 2020 at 21:49
  • $\begingroup$ @treskov How did you get that differential equation from Taylor expansion? $\endgroup$
    – thesecond
    Nov 7, 2023 at 15:55
  • 1
    $\begingroup$ @thesecond Via $$t^\prime(x)\mathrm dx = c\left(\frac{1}{f(x)}\right)\mathrm dx$$ and integrating both sides. Evidently $c_1=c$ and $c_2$ is the constant of integration. $\endgroup$
    – whuber
    Jan 21 at 16:23
0
$\begingroup$

The answers provided here are already useful. I just wanted to drop a couple more resources for learning about Box-Cox. There is actually an excellent episode from Quantitude dedicated to this very subject, which explains both the history and intuition behind this transformation (including the amusing history that the creators literally did this for the sole reason that their authorship on the paper would rhyme). I also would like to recommend this paper as a less math heavy article on the subject that shows examples of the transformation if you would like to see the practical side of the transformation in action. One of the useful things they do in this paper is show how different anchoring points affect the transformation, with an example shown below:

enter image description here

$\endgroup$
1
  • 2
    $\begingroup$ The story you quote is slightly mangled. Box and Cox decided to work together on a paper on transformations in the early 1950s when both were in Britain. It took some years before the ideas all came together. Box and Cox aren't just rhyming; there was a comic light opera "Box and Cox" in the 19th century. $\endgroup$
    – Nick Cox
    Jan 21 at 16:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.