I'm using support vector regression to model some fairly skewed data (with high kurtosis). I've tried modeling the data directly but I'm getting erroneous predictions I think mainly due to the distribution of the data, which is right skewed with very fat tails. I'm pretty sure a few outliers (which are legitimate data points) are affecting the SVR training, and perhaps also in the cross validation, where at the moment I'm optimizing the hyperparameters by minimizing mean-squared error.
I've tried to scale my data before applying SVR (e.g. using a sqrt function to reduce the outliers) as well as use a different hyperparameter minimization function (e.g. absolute error), which seems to give better results, but still not very good. I'm curious if anyone has encountered similar problems and how they approached it? Any suggestions and/or alternate methods most welcome.