Difference between Log Transformation and Standardization

Question

Is there any difference between the log transformation and standardization of data before subjecting the data to a machine learning algorithm (say k-means clustering)?

It looks like a common approach in preprocessing for clustering algorithms is to first un-skew the data through the log transformation and then perform standardization. My question is, don't both of these methods achieve the same effect when it comes to un-skewing the data? That is, both seem to transform the data into a normal distribution.

I do understand that standardization forces zero mean and unit variance but is it really required to perform both these methods on a single dataset?

So where do these two preprocessing techniques differ?

Answer 1

These two methods don't transform the data into normal distribution. And, they're very different.

• Standardization is just making the feature zero-mean and unit variance. e.g. if the feature is uniformly distributed, it'll again be uniformly distributed. It's just a linear transform, and it doesn't decrease the skew (i.e. skewness, which is already the third standardized moment).
• Log-transform decreases skew in some distributions, especially with large outliers. But, it may not be useful as well if the original distributed is not skewed. Also, log transform may not be applied to some cases (negative values), but standardization is always applicable (except $\sigma=0$).

The aim of stacking them together might be standardisation of all the features following the feature generation process.

Answer 2

• Standardization does not change the skew of the distribution. What it does is transforming the values so it has $\mu=0$ and $\sigma^2=1$. The actual distribution shape remains unchanged.

• Log-transformation, on the other hand, changes the skew of the distribution, and is useful when you deal with values that have right-tailed distribution.

---

Consider the following example.

set.seed(0)
X <- rlnorm(1000)
hist(X)

You can see that X has skewed distribution (long right tail).

Now we do two transformations as follows:

Z <- (X-mean(X))/sd(X)
L <- log(X)
par(mfrow = c(1,2))
hist(Z)
hist(L)

As you can see, standardization of X (which is Z in this example) has similar shape of distribution with X. Log-transformation of X (which is L in this example) is now transformed in a way that eliminates its long right tail.

Note that X in the previous example has lognormal distribution X <- rlnorm(1000), that's why the log-transformed variable looks normally distributed. If X is, say, exponentially distributed, the resulting log-transformed variable may not be normal, as shown below.

set.seed(0)
X <- rexp(1000)
par(mfrow = c(1,1))
hist(X)

Z <- (X-mean(X))/sd(X)
L <- log(X)
par(mfrow = c(1,2))
hist(Z)
hist(L)

You can see that the distribution of log-transformed X (denoted by L) now does not have right tail, but still not looking like normal distribution because it's still skewed. Log-transformation does not always make the distribution normal.

---