actually my study is about to choose best imputation techniques. Im comparing all the techniques using Root Mean Square Error (RMSE), Mean Aabsolute Error (MAE) and correlation (R). My problem is example imputation at Station . Method A has lower RMSE compare to method B and the R value is 0.7. Method B has lower MAE value and R value is also 0.7 Is it both method is consider the best?
Consider the difference between MAE and RMSE. While both aim to measure 'fit' in some sense, they have slightly different measures of what 'fit' actually is. RMSE penalises estimates further from the true value more than if a model produced several smaller errors, even though the total MAE may be the same.
For example see the below (python), where moving from
y_pred_2 improves RMSE but MAE is worse.
from sklearn import metrics from math import sqrt y_pred_1 = [6,4,3,2] y_pred_2 = [3,5,4,4] y_actual = [2,4,3,2] # MAE metrics.mean_absolute_error(y_actual, y_pred_1) # 1.0 metrics.mean_absolute_error(y_actual, y_pred_2) # 1.25 # RMSE sqrt(metrics.mean_squared_error(y_actual, y_pred_1)) # 2.0 sqrt(metrics.mean_squared_error(y_actual, y_pred_2)) # 1.32
Which of the measures is more useful depends on the specific task you're modelling. If it's unclear which metric is more useful, then I'd have said it's valid to report both, and try to understand why each of your imputation methods affects things the way they do.
Looking briefly at rainfall prediction literature, R^2 seems to be used (which is just your R value squared) at times e.g. Statistical models for predicting rainfall in the Caribbean. Although I also saw RMSE and MAPE used e.g. Evaluation of the Liu model for predicting rainfall. So no easy answer!