Can I transform my output variable in an imbalanced dataset?

Question

I have a dataset that has an output variable that is quite right-skewed and imbalanced. I want to use a neural network as a regressor to predict the output variable. Visually, it looks like there may be a function that I may be able to use to make the distribution roughly uniform, as the current distribution resembles a 1/x function or exponential function. Is there a robust way to transform this data into a uniform distribution? Or am I going about this in a totally incorrect way when something like under- or oversampling would be more appropriate?

Answer 1

If you want to ignore the prior induced by the distribution of your targets, you can just inversely weight the targets.

If they are discrete, then just count, if they are continuous, bin them in a representative way, and then optimize the loss: $$ L_\text{tot}(\theta) = \sum_{(x,y) \in D} \frac{L(f_\theta(x), y)}{c(y)} $$ where c(x) is just the count for that sample (bin/class count)

---

If you really want to be rigorous, you can think of your targets being a distribution, and you want to normalize by their likelihood: $$ \begin{align} L_\text{tot}(\theta) &= \sum_{(x,y) \in D} \frac{L(f_\theta(x), y)}{p(y)} \\ & =\sum_{(x,y) \in D} \frac{Z}{\tilde{p}(y)} L(f_\theta(x), y)\\ & \propto \sum_{(x,y) \in D} \frac{L(f_\theta(x), y)}{\tilde{p}(y)} \\ \end{align} $$

So, I would propose either to take the histogram you have and fit some polynomial, and then use that as a "score" ($\tilde{p}$), or fitting an actual distribution, using something like an exponential distribution, and also using that as a weight (these 2 methods should be "better estimate"/"smoother estimate" than an count/histogram)