I have 30 features in my self-collected dataset where I want to build a regression model. When I look at my data, most of the attributes (95% of the data points) are skewed on a very small range. Out of 30 features, only 2 features has sort-of normal distribution and other features are ranged within a majority range. Here is a graph of the distribution of some of the attributes:
My question is: is it safe/normal to work with such data? Or should I treat the 5% of the data as outliers and remove them, so that the rest of the 95% will have a normal-like distribution?
Intuitively, since majority of the attributes have very similar values, I am afraid that they won't have a distinguishing effect in the prediction. But I might be wrong, so I would like to ask here.