I am comparing error estimates from different models. I am looking at MAE (mean absolute error) and RMSE (root mean squared error) as my choice of error estimates. But the problem is that mostly they are not agreeing with each other.
Can there be any bias in the estimate, such that one of the error estimates will favour one type of model?
The RMSE gives higher weight to large errors as it squares before taking the average, does this work well even for the cases where dependent variable lies between [0-1]?
(There is a similar discussion on the netflix prize forum here.)
- I am little confused here, under specified circumstances can I choose a particular error estimate for identifying the best model ?