I have a regression problem and used different suitable models for predicting 4 dependent variables. Through many experiments, I realized that it's easy for many models to learn two tasks with high precision while the other two are less accurate.

That can be due to the less predictive nature of those two tasks in my datasets. Also in the context of my problem, it's not surprising each task has a different form of functionality. I mean, for each task $i$, the estimator $f_i(x) \approx y_i$ might have a different functional form. But I was wondering what other reasons might cause a model to perform on one task and not as good on another task.

A little info about what I've done:

  • standardizing the labels so that all are centered and have unit variance.
  • deployed linear regression, lasso, and neural nets as a model.
  • I also tried some feature selection as my problem is a very high-dimensional one.

UPDATE: After the same answer, I recalled that some of my regressors, despite being real numbers, are almost categorical, in the sense that their range only covers a few numbers (say integer numbers from 0 to 5) and one can in fact categorize the data by the value of this regressor. Now I'm wondering if that as well can explain part of the lower precision.


1 Answer 1


There are many possible reasons. Here's a start on a list, I'm sure others will add more:

  1. The model is misspecified
  2. The variables are measured with more error in one data set than another
  3. The relationship, while correctly specified, is less strong with some variables
  4. Some variables are measured on a coarser scale than others.
  5. Some variables have been categorized while others have not.
  • 1
    $\begingroup$ I appreciate your fast answer. I'll leave this question unanswered so that the community adds other reasons to your list. $\endgroup$
    – arash
    Commented Dec 10, 2019 at 15:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.