I have this strange condition. I have two predictors. One of the predictors has low correlation with the target but less rmse. On the other hand another predictor has high correlation but high rmse as well. But the number of samples for the two predictors are different. So the correlation and rmses are calculated on different samples sizes.
Can anyone explain why this is so?
I am venturing to guess that this is an instance of "missing not at random". The non-missing values associated with the predictor with low correlation seem to correspond with cases where there isn't much change in the dependent variable's values - which could explain why the rise is low and also that these values aren't very predictable, relative to the cases where the second predictor are non-missing. It would be instructive to create two dummy variables -- $m_1=1$ if predictor $x_1$ is missing, $0$ otherwise; and $m_2$ defined similarly. Then plot the dependent variable $y$ versus $m_1$ and $m_2$. My guess is that there is a notable difference between the missings and non-missings in at least one of these plots.
The scale could be off; do you have any shrinkage? If one predictor always predicts close to zero for a zero mean target variable, it could have small correlation but better RMS than a predictor that is well correlated but with twice the amplitude.
