Adjusting polynomial fit number of parameters

Question

I'm exploring a method to determine the optimal degree for polynomial regression in Python. Here's my approach:

1. I iteratively fit polynomials of increasing degrees (up to degree 50), calculating the adjusted R² for each fit.

2. I then fit a logistic function to these adjusted R² values to model how the goodness of fit changes with polynomial degree.

3. Finally, I identify the polynomial degree that produces the largest positive residual between the actual adjusted R² and the logistic model's prediction, and select this as the optimal degree.

Is this approach statistically sound? Are there more established or computationally efficient methods for determining polynomial degree in regression analysis?

The polynomial seems to behave well, I'm using it to generate trends on time-series data.

They don't seem to have much predictive power, in my opinion, due to the high degree of parameters, but they may be useful for analyzing historical trends.

Answer 1

There might be a specific duplicate of this question on the site, but until that's identified here's a brief answer.

This approach is not statistically sound, for a couple of reasons at least. It's also not clear that it's any simpler (let alone better) than LOWESS, if all that you want is a smooth summary of historical data.

First, it uses high-degree polynomials. Forcing a single high-degree polynomial to fit over an entire range of data typically won't work well unless there is a strong theoretical reason for the polynomial. There are much better choices, like regression splines or generalized additive models, to handle nonlinear associations between a predictor variable (evidently time, here) and an outcome variable within linear regression. That's true even if you just want to describe historical trends rather than make predictions.

Second, it uses the result of trial regressions to determine the ultimate form of the regression. That leads to the multiple problems involved in automated model selection, including severe overfitting that makes results difficult or impossible to generalize beyond the data set at hand. There are ways to penalize regression coefficients to minimize overfitting, but your proposed approach didn't specify penalization. I suppose that's less of an issue if all you want to do is describe the historical data, but I suspect that your attempt at high-degree polynomial fitting might have itself led to the limited "predictive power" that you found. You might have better predictive power if you use methods like those covered in Forecasting: Principles and Practice.