# Finding degree of polynomial in regression analysis

I am working on a machine learning project where I am trying to fit a curve on data. Unfortunately the date has somewhat high feature vector. So, I can't really plot them on a 2D or 3D space to guess how the shape of the data looks like.

So, other than hit and trial, does there exist a mathematical way to find degree of polynomial that might best fit my data.

I mean I do know that I can look into a least square error for each degree and then choose the one with the minimum error, but then it will be a double optimization problem as the first optimization loop would consist in finding a set of weights for the curve that fits the data, while the second loop would be used to check the degree. Any suggestions?

• Are you planning on doing any statistical inference (confidence bands, hypothesis tests, etc.)? That would alter the approach. Commented May 1, 2015 at 12:09

Sorry if this is too elementary, I just wanted to make this answer as self-contained as possible. In fact, you can't do what you're describing: the best polynomial of degree $k+1$ will always fit at least as well as the best polynomial of degree $k$, since the set of $k+1$ degree polynomials includes all $k$ degree polynomials (just set $a_{k+1} = 0$). As you continue to increase $k$, at a certain point you will be able to find a polynomial that fits the data perfectly (i.e. with zero error).