1
$\begingroup$

I'm working on a problem where I need to fit a regression on solubility data from a collection of molecules. The response variable (solubility) displays a bimodal distribution, suggesting there are different populations in my data:

enter image description here

After training a baseline random forest model, I plotted the regression line and the bimodal nature of the target variable is pretty visible:

enter image description here

However, when I inspected the residuals I noticed the were centered around 0 but still showing the bimodal distribution

enter image description here

I'm not sure what would be a sound approach here (its my first time seeing this kind of distribution in this particular type of data).

  1. I read in here that regressions don't make assumptions about the distribution of the target variable. Does that apply for all algorithms (e.g. PLS, random forest etc)?

  2. I was following this kernel and it seems the author was trying to separate the populations. Is this a possible solution if a regression is required? My intuition is that we could use the classifier to predict from which population a sample comes from and then use the equivalent regression model to make predictions.

  3. Is the bimodal pattern in my residual plot worrying or I should only be concerned if they are centered around zero?

$\endgroup$
1
  • 1
    $\begingroup$ As Dave said (+1) you are mostly on the clear in terms of bimodality. For that matter, if our population is truly bimodal (e.g. men's and women's height distribution in cm) then it is completely normal to have clusters/lumps in our residuals. That said, it is unclear what exactly that residuals plot shows, the one we care about the most is residuals versus fitted values rather than residuals versus observed values; the errors should be independent of the predictors $X$, not the data $y$. $\endgroup$
    – usεr11852
    Apr 5, 2022 at 17:21

1 Answer 1

4
$\begingroup$

Your residuals, as I judge from the histogram, have a normal-looking distribution. You’re noticing that the predictions have bimodality, which is a different claim.

I don’t want to claim that there is no regression method that cares about the distribution of the predictions, rather than the residuals, but I can’t think of one.

What concerns me is that your residual-actual plot seems to have a decreasing trend. For instance, the residuals in the $0-50$ $x$-axis range are a bit higher than those in the $200-250$ range, as histograms of those subsets are likely to reflect. You want the residuals to be pretty consistently centered around zero, not indicating overestimates in one region and underestimates in another.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.