Building a predictive model, regression with a long right tail

I am trying to build a, regressive, predictive model for a target time-series that is heavily skewed.

You could think of the target as being like earthquake magnitudes or heavy rainfall. Most of the time we sit in the relatively boring head of the distribution, but we want to predict the interesting 'extreme' events.

The distribution of target values looks like this (in a histogram with Bayesian block sizing)

Are the approached legitimate alone or in combination?:

1. Predict in f(log(target)) space. Where f(x) is used to produce a zero-mean, unit-variance distribution.
2. Prefer non-linear (e.g. tree based, Support-Vector-Regressors with non-linear kernels) estimators.
3. In choosing samples for learning, validation and testing: relatively oversample the right-tail of the target distribution.

Is there something else I should try?

In case it helps with context I'm using python and sklearn.

• The two modelling methods you propose seem to me to be legitimate in principle, but obviously the best model will emerge from analysis. Modelling the data under a logarithmic transformation (assuming you have no zero values that would stuff it up) implicitly gives you a non-linear model anyway, so that is fine in principle. As far as the decision to split the data for training and testing, I would suggest that rather than oversampling a particular part of the distribution, you should instead be explicit about a loss function for getting things wrong (maybe higher loss for high end?). – Reinstate Monica Jan 25 '18 at 3:07