I have approximately 3000 products for which I have to forecast in every, say, 2 months. I have the code in place for different forecasting models such as ARIMA, forced seasonal ARIMA, STLF etc.
Now for each product, I have forecasts coming out from 8 different models. Currently I use MAPE as a parameter to decide which forecast is the best. But sometimes I get a forecast which is a complete straight line and still has the least MAPE. These kinds of forecasts(straight line ones) don't really help me in making any decision. I also want the forecasts to capture seasonality which models like forced seasonal ARIMA do. But when I'm looking at the results of all the 3000 products, I do not have a perfect metric in place to see whether the output with best MAPE captures seasonality or not.
Is there a parameter which I can use, which can try to quantify the seasonality and MAPE together in the output and help me make a better decision of choosing the most appropriate model?
An example: I am forecasting for products (weekly, for 52 weeks) for, e.g., sales. I need to know how many will I sell each week. But if I get a flat line as my best output, I would not know exactly how much will I sell each week. This will stop me from estimating accurately how much space is required to keep these products. But if the 2nd best model, that has a little higher MAPE, but captures seasonality, then it enables me to decide that, okay, I need 'x' shelves to keep these products.
So these kind of small decisions, are more accurate if the seasonality is captured in the output. So I need to find a middle ground somewhere, a combination of some sort of MAPE and seasonality or any other metric which can help in deciding such things.