# Correlation between prediction error and regression dependent variable

I've trained a regression model which predicts the dependent variable from several independent variables. I noticed that there is a strong negative correlation (-0.86) between the dependent variable and the error (prediction - dependent variable). What does this imply?

While it can be caused by all predictions being similar, therefore, creating a linear relationship between the error and the dependent variable, there is also a positive correlation (0.47) between the predictions and the dependent variables.

Is there a way to reduce the errors for larger dependent variable values?

• It is a necessity that $\hat{Y} - Y$ be correlated with $Y$. What you don't want is a correlation between $\hat{Y} - Y$ and $X$ or $\hat{Y}$ as this indicates systematic lack of fit. Jan 16 at 13:48

## 2 Answers

Look into class imbalance or skewed datasets. Since your dependant variable is imbalanced your accuracy will be biased towards the small values. I'd recommend trying an ensemble algorithm like Bagging. Techniques like oversampling/undersampling might also produce improvements.

The fact that the error is correlated with an independent variable may mean that there is an unobserved variable that is both relevant to explain the dependent variable and related to the independent variable - can you maybe figure it out? I.e. try to predict ice cream sales with the amount of snow that falls - they are correlated, but actually both variables relate to temperature.

In the meantime, you could also use transformations of the dependent variable, like its power:

y = beta_1 + beta_2 * x + beta_3 * (x²)