Suppose I have a collection of MAE across multiple time series (say, 10), and 3 models. However, MAE cannot be compared across time series.
I compare the errors in this way: assign ranks to models, so that each model gets a rank for every time series with respect to its MAE. As a results, for each model instead of MAE, I have a collection of ranks: 10 ranks for each model, where ranks are from 1 to 3.
Can I analyze this qualitative "rank error"? If I analyze it, what assumptions should I make?
NB. Some information is lost in the calculation of ranks, since ranks 1 and 2 for time series 1 can be divided by 0.1 in MAE (insignificant), while the same ranks can be divided by 1000 in MAE for another time series (which is exceptionally significant).
