# When would you need scaled error between different time series evaluations?

Let's say we have 3 time series for three different fruits sales over one year. Although they are all fruits, their daily sold volume is very different.

For example, imagine apple being sold consistently from day to day and in general the volume is large (e.g. between 100 to 1000 daily apple sold), vs some other rare seasonal fruit (most days zero sales, and on a few days of the year, there might be ~5 sold).

Now the task is to build model (single or one for each?) to predict the sales in the future, and evaluate how well the model performs.

For model training (e.g. using sequential model like an RNN), I would need to normalize the data, I suppose I could either normalize it within each time series (so apple's time series divide by it's max sales for example) or normalize it over all three. For inference, I'd need to multiply the normalization factor back to get the actual sale prediction.

To evaluate the model, by default I'm thinking to use MAE or RMSE on the sale prediction, but it's obviously scale dependent, and then I read in 3.4 Evaluating forecast accuracy, it seems that I could use scaled error instead (it basically divides over naive prediction, which carries the scale so the end value is sort of scale free), then I can combine or compare the 3 different time series' performance.

My question is that, since I normalized the training time series to all between 0 and 1, can't I just compare the 3 time series using rmse on raw predictions (i.e. without multiplying the normalization factor back)? If so, would the scaled error listed in above textbook still be relevant somehow?

• I'm sorry, I'm not sure I understand your question. Are you asking A) Whether you should train a single model for all, or separate model for each series? or B) How to compare prediction errors between the time series (regardless of prediction method used)? Jan 20, 2022 at 3:19
• The main question is B, although I would appreciate any opinions on A if that's possible, thanks!
– Sam
Jan 20, 2022 at 14:27