I predict sales for a 1000 products. I use time series cross validation to evaluate forecast for each product.
I want to report how well the model is performing over the whole product range. Thus, the ideal summary would be 1-3 numbers that are easy to interpret.
Product sales vary significantly. Some product are fast sellers with many sales per day. While others are very slow sellers - zero sales for periods of time.
I looked at:
- MAE, but it suffers from different scales,
- MAPE, but it's unstable for slow sellers,
- MASE, but it's hard to interpret and I'm not sure if I can aggregate it.
What metric would be best to capture an overall performance of the model? Or should I split data into fast/medium/slow sellers?