# Should I apply log transformation to column with long-tail distribution before clustering? [closed]

I am doing clustering on a given data. When I plot the distributions of the individual features of this data, I found there are many columns that shows "long tail distribution".

I am wondering should I apply log transformation to these columns ?

If yes, what is the reason to do this?

I tried to conduct feature scaling only, which I would use $$\frac{X−X.mean()}{X.std()}$$. This should transform the values to be around 0. But I worry after the feature scaling the distribution is still so skewed that, it may be not useful to the clustering task? So I'm wondering if I should apply the log transformation before the feature scaling (standardization).

## ---------------------------------------------------

This is what happens when I apply log transformation: • I do not know much about your intention for clustering, but such transformation is often useful for numerical reasons, also for results presentations, outlier detection, etc. Maybe also to find a law for the tail behavior. If the clustering is e.g. regarding distances such transformation can also help. Jan 30 '19 at 12:18
• My objective of clustering is to segment the customers into two types using the purchase/behavior data. My initial intention for the log transformation was, taking the above part of data as an example, the majority of values falls below 100, meantime there are few values are between 100 and 700, some of which may be considered as outliers. If I conduct feature scaling like normalization or min-max scaling, the majority of values between 0 and 100 will be squashed into a very small range due to the existence of the few large values. I'm not sure if that's a problem for clustering. Jan 30 '19 at 13:48
• Usually you would standardize your variables so that you eliminate the effects of huge variances. Which is what you've done here, in a sense. So it might work, only one way to find out. Jan 30 '19 at 15:09
• @user2974951 I think you meant feature scaling, which I used $(X-X.mean())/X.std()$. But I worry after the feature scaling the distribution is still so skewed that, it may be not useful to the clustering task? So I'm wondering if I should do the log transformation before the feature scaling (standardization). Jan 30 '19 at 15:48
• @mkt Thank you for the comment. I'm wondering is there any article about the relations between clustering algorithms and data distributions? Thanks. Jan 31 '19 at 5:16