# Given loads of data, can we always model it with polynomials?

Given Taylor series and enough data so as to not risk over-fitting, do you actually need to think about if your phenomenon is following an exponential, quadratic, logarithmic, ..., behaviour? I'm sure you can come up with counter-examples of when this isn't a good idea, but if we're very general and pragmatic, is it usually 'fine' to just fit the data to some degree nine polynomial hoping whatever bizzare pattern is hiding in the data will be very well approximated by its power series?

• +1 for an interesting question. But you seem to expect much more out of Taylor series than they can possibly supply. I would anticipate good answers variously to appeal to parsimony, interpretability, and predictability and to suggest alternatives such as splines. Others might be moved to discuss how modeling the variability of the response can be equally or even more important in many applications. – whuber Jul 28 '14 at 21:26
• Rather than necessarily focus on polynomials per se, there's a wide literature on approaches that don't try to identify a specific functional form but focus on assuming smooth relationships. Both spline-regression (including penalized-splines) and local-linear/local-polynomial (i.e. kernel-based) smoothing are motivated by this sort of desire. See, for example, chapters 5 and 6 of Elements of Statistical Learning 2ed Hastie, Tibshirani, & Friedman (a pdf is available at the book website there - you can see the link by scrolling down). – Glen_b Jul 28 '14 at 23:57
• Maybe the function you're trying to model isn't differentiable. For example en.wikipedia.org/wiki/Psychological_pricing – hahdawg Aug 28 '14 at 0:18

Imagine replacing an arbitrary model with polynomial parameters with a series of dummies for all values of the explanatory variables and all their interactions. If you have enough experimental data that's going to be as general as possible. The highest order polynomial function of all the interactions is going to fit the data as well. So if you have the data and setting to estimate the fully saturated model that will be model free.

For example, consider that the true model is where X and Y take dummy values only:

Z = F(X,Y)

You could also write this model as:

Z = beta0 + beta1 * 1(X) + beta2 * 1(Y) + beta3 * 1(X) * 1(Y)

These functions match everywhere on the values taken by X and Y. This gets to be quite thorny if you have multiple parameters and they take multiple values, but the principle is the same.

I learned about this from Mostly Harmless Econometrics (p. 48 - 51) where they argue that the case of saturated modeling implies that linear modeling is equally general as non-linear modeling. Moving into non-saturated models means that non linear models can cover functions with fewer parameters, but with enough free parameters and the data to estimate them, they cover the same set of models.