# How to remove skewness from the population?

Just want to know the ways to remove skewness from the skewed population. I am working on logistic regression (in the insurance domain) for finding out the repurchase propensity of existing users. Income data is right skewed. Will it cause any problem for runing logistic regression? If yes then how to remove it (deletion or any other transformation)?

-
This is a very terse question. You need to expand and describe (i) why you think skewness is a problem; (2) how bad skewness in your data is; (3) whether these data are really population (all available objects) or a sample (several from a myriad possible); (4) your computing platform -- you've received a default R answer that you may be unable to utilize unless you are familiar with the package. –  StasK Aug 18 '11 at 16:02
@Stas is right: our FAQ encourages you to pose "practical, answerable questions based on actual problems that you face." We're here to help with specific problems (but not to write monographs on general techniques!). Give us a context and some details so we can provide you some really effective answers. –  whuber Aug 18 '11 at 16:07
how many dimensions? –  user603 Aug 18 '11 at 16:54
@statsK & whuber : You people are right, I should have mentioned the entire context. I am working on logistic regression in insurance domain for finding out the repurchase propensity of existing users and income data is rightley skewed. Will it cause any problem for runing logistic regression? If yes then how to remove it(deletion or any other transformation)? –  anujk Aug 19 '11 at 9:27

A flexible framework for this is provided by the Box-Cox transformation, i.e., a power transformation $y \mapsto C(y,\lambda) y^\lambda$ where the parameter $\lambda$ of transformation $\lambda$ is driven by the data (essentially, towards reducing the skewness); $C(y,\lambda)$ is a scaling parameter to make comparison of the goodness of fit provided by the different powers meaningful.

-
I thought in that setting $\lambda$ was chosen to maximize the likelihood, not to minimize skewness? Perhaps the two goals are the same when a normal likelihood is used - is that the case? –  Macro Aug 18 '11 at 16:31
@macro I don't know if the two goals are the same when a normal is used, but you can pick $$\lambda$$ to minimize skewness if you want. –  Peter Flom Aug 18 '11 at 20:02
Ok. In my experience $\lambda$ is chosen based on its maximized profile likelihood - I suppose I could imagine choosing it with some other criteria in mind. –  Macro Aug 18 '11 at 20:05
@Macro: you are right, it will be a transformation towards normality only if you put it into a normal likelihood. Frankly, though, I have not seen it used otherwise. I admit my answer was incomplete, so I will edit it to make it more technically accurate. –  StasK Aug 19 '11 at 3:58

There's no way to remove skewness from the raw data set without chopping off the tail (i.e. deleting all of the observations that make it "skewed"). In regression it is common to transform the data set so to eliminate skewness in the residuals. If the data is non-negative then the $\log$ or $\sqrt{\cdot}$ transform are the default choices.

Using data from the exponential distribution (a skewed distribution) as an example, each of these choices do reduce the skewness, but don't take care of it entirely (see R code):

 x = rexp(1000,rate=4)
plot(density(x))
plot(density(log(x)))
plot(density(sqrt(x)))


In the case of the log-normal (another skewed distibution), the log transform makes it perfectly symmetric (and normally distributed), since that is how the log-normal is defined. You might try playing around with some other transformations if eliminating skew is important for your application.

-