0
$\begingroup$

I have this time series I want to perform polynomial regression on, to estimate the trend. data

To start, I tried using only a second order polynomial, these are the results (AIC=30.37105) fit R output

We can see how the curve doesn't fit really well, mostly in the center. Checking the residuals after the seasonal component has been estimated (using four dummy variables, one for each season, added to the previous linear model), it is obvious that there is still some auto correlation. residuals

The auto-correlation function shows that the residuals don't seem like only white noise. acf

Then I tried using a fourth order polynomial. (AIC=27.60569) fit R output

We can see that adjusted R^2 has increased and that all the coefficient are still significant (not as significant as before). These are the residuals, again, after the seasonal component has been estimated using four dummies added to the model above residuals

And the auto-correlation function on them acf

This seems much better to me. (AIC for the second model with the dummies is lower too)

However one of my teachers at University told me that it makes no sense to use a polynomial of order four in a situation like this because the second model is surely overfitting. Please note he still told me that the model of order 2 isn't good either (by looking at the residuals), and we will see better ways to handle time series, but I still thought the model of order 4 was an improvement.

Things I don't understand:

  1. How can I detect that the second model is overfitting? (Is it? I would say that it is fitting better, mostly looking at the residuals and AIC)
  2. Why is it always suggested not to use polynomials of order higher than 3-5? (Thinking that polynomial regression isn't that useful for out of sample extrapolation anyway, I would say that at least I should try to create a model that fits the data in sample well)

The time series: y=c(9.328161, 9.262356, 10.23992, 9.937655, 10.90888, 11.055, 11.09276, 10.51899, 11.59662, 11.98787, 11.66901, 11.30161, 11.60235, 11.7764, 11.8741, 11.24226, 11.70456, 11.98945, 11.99912, 11.15315, 11.82424, 11.88189, 12.11866, 11.44164, 11.92071, 11.888770, 11.6900, 11.10875, 11.08220, 11.37809, 10.84132, 10.100, 9.998741, 10.05856, 9.600)

$\endgroup$

1 Answer 1

2
$\begingroup$

I think when you're instructor introduces more-time series modeling methods you will get a better intuition for the problems with your models in general.

There are a few points to be made about the limitations of your models:

  1. Autocorrelation. Simply put, when you have auto-correlated data, linear models simply aren't appropriate. They are ALL biased, because they assume no autocorrelation.

  2. Overfit. Overfit cannot always be detected just by looking at a model. This is one of the reasons we often hold out some data for validation by training on some data and then testing it against held-out data. This allows us to detect overfit. So, you're right, there isn't a perfect quantitative way per se to detect overfit, because by definition overfit is related to the true distribution across the population, which you haven't observed.

  1. Polynomials. In general higher-order (e.g. 3-5) polynomials are discouraged in most fields because they are postulating that there is some underlying mechanism that has a high order function. This simply doesn't happen very often outside of physics. In other words, this is a theoretical objection, not a empirical objection. Unless you can substantively justify the use of a high-order polynomial, it will always be viewed as skeptical. And, with validation data, you would most likely see this is the case.

Another way to think about this: you stated: "I would say that at least I should try to create a model that fits the data in sample well." I disagree with this statement. I think the goal of a statistical model is to explain the data sample well. If we cared only about pure predictive power, then most likely you would be better off with a machine learning algorithm like random forest, SVM etc. You use statistical models to understand, not just predict.

$\endgroup$
2
  • $\begingroup$ Thank you for your answer. I'm not really getting your last statement: when I stated that I wanted the model to fit the data "in sample" well, I meant I want to explain the underlying trend well (in sample), so I agree that the goal is not prediction in this case (and that's why I thought an higher order polynomial wasn't a problem). However your third point did satisfy some of my doubts: if the underlying trend isn't an high order one, it doesn't really make sense to use an high order polynomial (do you have some suggestions on readings to look at about this statement?). Thanks for your time $\endgroup$ Oct 15, 2021 at 19:47
  • $\begingroup$ There are some good blog posts on this that give good intuitions: statisticsbyjim.com/regression/overfitting-regression-models $\endgroup$ Oct 15, 2021 at 19:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.