I'm trying to create a regression model using data that are heavily skewed, almost like a half-normal distribution with a mode at zero. The lower scores are over-represented due to sampling technique - my outcome of interest (the x-axis in the histogram, the y-axis in the scatter plot) is only present in the minority of this population. I'm trying to use this model to predict outcomes in a different population which will not be skewed in the same manner. Ideally it would be great if I could drop some of the low scores in order to normalize the distribution somehow, but I can't find a validated approach to undersampling a continuous distribution. It would also be nice to be able to transform the data somehow, but that just doesn't seem to make sense because the low scores will still be overrepresented and would still drive the model.
So the zeros (and other low values) end up driving my model (see below). In the real world, I think that the relationship is only relevant when y>1.
Just for fun, I tried dropping all of the data with y<1, and it worked beautifully well for predicting outcomes in two independent validation datasets. Of course, I can't just drop half the data without some sort of strong statistical justification. If I don't drop any data, I can still predict outcomes in one of my validation datasets, but it doesn't quite work in the other one (p=0.1). I tried using various transforms (log, beta, log-beta, rank, square root, and Box-Cox), and they all give me the same issue - the histogram looks less skewed after transformation, but the effect is still driven by the over-represented low values.
Any suggestions for how I could approach this?
Some side notes that may or may not be relevant:
*I also have a handful of covariates which have a small impact on the results, but I didn't include them in the above info because I'm not sure if that's relevant here.
*I'll be using MATLAB to repeat the analysis across thousands of predictors and then use a permutation test (thousands of permutations) to confirm significance, so it would be nice if any proposed solution could be implemented in MATLAB. If that's not possible, that's OK - I can figure out a way to switch back and forth.
EDIT: I understand that I could undersample the over-represented part of the distribution. The problem is that my outcome is continuous, so I can't just "undersample the major variable" without arbitrarily choosing a cutoff between the "major" and "minor" variables. This would introduce bias, since I'm choosing a cutoff after looking at the data. I was hoping I could find a validated algorithm that automatically chooses which parts of the distribution to undersample.