Newbie question, sorry. I have a highly seasonal monthly time series, predictable with no exogenous/independent variables and no obvious trend. I want to show that a suitable state space model (using
dlm) is a better predictor than both Holt Winters and a SARIMA model and that some more modern methods (single layer neural network
nnetarfrom the R package
XGBoost) are better still.
1) I have split my data 80/20 train/test, what single measure should I use to establish a comparative measure of predictive power across all these methods? A data science competition I saw recently used sum of log squares:
Is this appropriate across all methods here? I've not seen the log correction before, what's the purpose of this?
2) I have seen some literature suggesting a 60/20/20 split of data to allow for cross validation. Why wouldn't I just fit the model to 80% of my data and use part 1) to select the best method?