I have two models/indices that try to predict observed values. I've compared them using correlation and regression, but I'd like to use MAE (Mean absolute error) to asses which of them is more closer to the observed values. The problem I ran into, is that they use different scales.
The observed values range
from 0-100, the first model has the range of
0-100 as well, but the second one produces results that range from
-1 to 1. This is due to the first model actually trying to predict the phenomenon collected in the observed values. The second model is trying to predict another phenomenon, but I'm trying to test whether it is also suitable for predicting the observed phenomenon. They both use different data and it would be difficult to standardize the data before running the model, because they so methodologically different.
I tried normalizing the observed and predicted values into z-scores with
z = (x - mean(x))/ stdev(x) where x is a raw value, mean(x) is the mean of x population and stdev(x) the stdev of x population
Then calculated MAE with
sum(abs(obs_z-pred_z)/count(obs) based on the z-scores. I got some values from this, but I'm not sure whether this a good method.
I also tried testing SMAE (Standardized mean absolute error) with both
mean(abs(obs-pred)) / mean(obs) and
mean(abs(obs-pred)) / stdev(obs) and these produce relatively similar results to eachother, but completely different to the z-scored MAE.
So therefore, is there a good or established method for comparing errors of models that produce values different scales? Ideally I'll have more of these models/indices in the future. Therefore, it would be good to settle on one working method because most likely some of them will vary in scale. I'd also like to use RMSE in a similar fashion to MAE if thats possible. Or is there a totally different or better way for doing the comparison? I'm definitely not that familiar with the world of statistics so any help is appreciated.