Which if these two models works better?

Question

I have this time series I want to perform polynomial regression on, to estimate the trend.

To start, I tried using only a second order polynomial, these are the results (AIC=30.37105)

We can see how the curve doesn't fit really well, mostly in the center. Checking the residuals after the seasonal component has been estimated (using four dummy variables, one for each season, added to the previous linear model), it is obvious that there is still some auto correlation.

The auto-correlation function shows that the residuals don't seem like only white noise.

Then I tried using a fourth order polynomial. (AIC=27.60569)

We can see that adjusted R^2 has increased and that all the coefficient are still significant (not as significant as before). These are the residuals, again, after the seasonal component has been estimated using four dummies added to the model above

And the auto-correlation function on them

This seems much better to me. (AIC for the second model with the dummies is lower too)

However one of my teachers at University told me that it makes no sense to use a polynomial of order four in a situation like this because the second model is surely overfitting. Please note he still told me that the model of order 2 isn't good either (by looking at the residuals), and we will see better ways to handle time series, but I still thought the model of order 4 was an improvement.

Things I don't understand:

1. How can I detect that the second model is overfitting? (Is it? I would say that it is fitting better, mostly looking at the residuals and AIC)
2. Why is it always suggested not to use polynomials of order higher than 3-5? (Thinking that polynomial regression isn't that useful for out of sample extrapolation anyway, I would say that at least I should try to create a model that fits the data in sample well)

The time series: y=c(9.328161, 9.262356, 10.23992, 9.937655, 10.90888, 11.055, 11.09276, 10.51899, 11.59662, 11.98787, 11.66901, 11.30161, 11.60235, 11.7764, 11.8741, 11.24226, 11.70456, 11.98945, 11.99912, 11.15315, 11.82424, 11.88189, 12.11866, 11.44164, 11.92071, 11.888770, 11.6900, 11.10875, 11.08220, 11.37809, 10.84132, 10.100, 9.998741, 10.05856, 9.600)

Answer 1

I think when you're instructor introduces more-time series modeling methods you will get a better intuition for the problems with your models in general.

There are a few points to be made about the limitations of your models:

1. Autocorrelation. Simply put, when you have auto-correlated data, linear models simply aren't appropriate. They are ALL biased, because they assume no autocorrelation.

2. Overfit. Overfit cannot always be detected just by looking at a model. This is one of the reasons we often hold out some data for validation by training on some data and then testing it against held-out data. This allows us to detect overfit. So, you're right, there isn't a perfect quantitative way per se to detect overfit, because by definition overfit is related to the true distribution across the population, which you haven't observed.

3. Polynomials. In general higher-order (e.g. 3-5) polynomials are discouraged in most fields because they are postulating that there is some underlying mechanism that has a high order function. This simply doesn't happen very often outside of physics. In other words, this is a theoretical objection, not a empirical objection. Unless you can substantively justify the use of a high-order polynomial, it will always be viewed as skeptical. And, with validation data, you would most likely see this is the case.

Another way to think about this: you stated: "I would say that at least I should try to create a model that fits the data in sample well." I disagree with this statement. I think the goal of a statistical model is to explain the data sample well. If we cared only about pure predictive power, then most likely you would be better off with a machine learning algorithm like random forest, SVM etc. You use statistical models to understand, not just predict.