Skewed data for regression analysis I have 30 features in my self-collected dataset where I want to build a regression model. When I look at my data, most of the attributes (95% of the data points) are skewed on a very small range. Out of 30 features, only 2 features has sort-of normal distribution and other features are ranged within a majority range. Here is a graph of the distribution of some of the attributes:

My question is: is it safe/normal to work with such data? Or should I treat the 5% of the data as outliers and remove them, so that the rest of the 95% will have a normal-like distribution?
Intuitively, since majority of the attributes have very similar values, I am afraid that they won't have a distinguishing effect in the prediction. But I might be wrong, so I would like to ask here.
 A: First thing I'd suggest is that it looks like the huge spikes are zero. What do the graphs look like if you exclude zero? There is a class of problem called Zero Inflation Models where you have a lot of zero values and need to distinguish between zeros during times when non-zeros are possible, and zeros that are during times where non-zeros aren't really possible.
For example, if you're looking at the network traffic for desktop computers in a business, you might find that people turn off their machines (or the machines go to sleep) outside of normal business hours, so you can't get non-zero usage for those machines during those hours. During the day, you might see zero usage, but it's more random and depends on what the user is doing.
Second, are you concerned about extreme values (for say bandwidth provisioning)? There is an entire field of statistics called Extreme Value Theory, from which you may find the Weibull distribution useful.
A: I would recommend keeping all data. As @COOLSerdash notes, the inputs to a regression don't need to be normally distributed, the residuals do (in order to do certain statistical tests - if you don't care about inference, you can disregard normality).
The long tails may be just the input data values that are most important to your dependent variable! Look at some transformations, e.g., logs or Box-Cox, and use cross-validation to choose the best predictive model.
A: I would suggest exploring some common data transformations such as square-root and log transforms. Examine the normality of each and remember which ones you have used when interpreting your coefficients. In terms of outliers, there are quite a few approaches to how to handle them. One thing you can always try when there are a few outliers is run the regression with and without them...and compare the results. 
