I am adjusting historical monthly income amounts to an ARIMA model using auto.arima on R, with forecasting purposes.
For that I have gotten different models based upon different time windows of the same data. For example model "arima.15Y" was constructed using last 15 years of information, model "arima.10Y", was constructed using last 10 years of the same sample, and so on.
Since information criterions can´t be used to compare models with different sample size, I am justifying model selection (lastly, time window selection) by comparing the mean squared-error of the adjusted data against the observed data for a)the whole adjustment frame for each case) and b)the last year (since I´m mainly interested in 1 year forecasting).
My question is, if my reasoning seem correct and logical and if there is a more formal way to justify time window selection.