I have a regression problem and used different suitable models for predicting 4 dependent variables. Through many experiments, I realized that it's easy for many models to learn two tasks with high precision while the other two are less accurate.
That can be due to the less predictive nature of those two tasks in my datasets. Also in the context of my problem, it's not surprising each task has a different form of functionality. I mean, for each task $i$, the estimator $f_i(x) \approx y_i$ might have a different functional form. But I was wondering what other reasons might cause a model to perform on one task and not as good on another task.
A little info about what I've done:
- standardizing the labels so that all are centered and have unit variance.
- deployed linear regression, lasso, and neural nets as a model.
- I also tried some feature selection as my problem is a very high-dimensional one.
UPDATE: After the same answer, I recalled that some of my regressors, despite being real numbers, are almost categorical, in the sense that their range only covers a few numbers (say integer numbers from 0 to 5) and one can in fact categorize the data by the value of this regressor. Now I'm wondering if that as well can explain part of the lower precision.