Let's say we have 3 time series for three different fruits sales over one year. Although they are all fruits, their daily sold volume is very different.
For example, imagine
apple being sold consistently from day to day and in general the volume is large (e.g. between 100 to 1000 daily apple sold), vs some other
rare seasonal fruit (most days zero sales, and on a few days of the year, there might be ~5 sold).
Now the task is to build model (single or one for each?) to predict the sales in the future, and evaluate how well the model performs.
For model training (e.g. using sequential model like an RNN), I would need to normalize the data, I suppose I could either normalize it within each time series (so apple's time series divide by it's max sales for example) or normalize it over all three. For inference, I'd need to multiply the normalization factor back to get the actual sale prediction.
To evaluate the model, by default I'm thinking to use MAE or RMSE on the sale prediction, but it's obviously scale dependent, and then I read in 3.4 Evaluating forecast accuracy, it seems that I could use
scaled error instead (it basically divides over naive prediction, which carries the scale so the end value is sort of scale free), then I can combine or compare the 3 different time series' performance.
My question is that, since I normalized the training time series to all between 0 and 1, can't I just compare the 3 time series using rmse on raw predictions (i.e. without multiplying the normalization factor back)? If so, would the scaled error listed in above textbook still be relevant somehow?