I am also attempting to automate the forecasting of financial parameters of multiple products in the Insurance sector. For this, I am planning to use the
About my data:
- Timeseries having 50 data points (monthly data for 4 years + a quarter)
- Some products begin a little later than the others and therefore have lesser observation points.
- Usually, the data of all the product is seasonal.
- There are no external parameters that have to be taken into consideration and only historical data is to be used for forecasting.
My question is more concerned with the pre and post forecast steps i.e.,
- Data prepartion and
- Model fit/accuracy
In the automation code, I would want to create a chunk which does an automated data preparation, as needed by the product. Now, is there something similar to the forecast package which automatically does the data transformation too? I have tried the
forecastfunction, but am not getting very good results.
If not, could anyone suggest, what must the bare minimum tests be? For example, I see that
ndiffwill help us in identifying if differencing is needed and the level too.
As far as checking the model accuracy is concerned, I want to understand that when we use the ets function or forecast, the models created for each contract will have certain parameters and that would be used for the test data. But, do we need to do a model fit on the test data in this case too? The MAPE or RMSE value of each product's forecast (for train data), is that not sufficient? I am not wishing to divide my data because of the lesser no of observation points.
Is it advisable to completely remove all those products from the analysis which have lesser datapoints or have a bad quality ( like, maximum observations= 0). If I include them in the analysis, would it effect the analysis of other products?
Really looking forward to a guidance.