According to Wikipedia, curve fitting is defined as:

the process of constructing a curve, or mathematical function, that has the best fit to a series of data points

And function approximation is defined as:

select a function among a well-defined class that closely matches ("approximates") a target function in a task-specific way

Specifically, I am talking about the context of a deep neural network, which are often described as function approximators. But according to the above definitions, I do not see the difference between describing a network as a curve fitter, and a function approximator. Both attempt to learn a function which matches the observed data as closely as possible. What is the key difference?

  • $\begingroup$ First, those are very general definitions. Second, you can approximate function by other things then a curve -- so you have the most basic difference. $\endgroup$
    – Tim
    Oct 18 '16 at 12:40

They are very similar, and different people in different community may have different definition. The following answer is based on my understanding.

  • In numerical analysis framework, "curve fitting" is often used to describe interpolation, where the ultimate goal would be trying to minimize the "training loss", i.e., the loss for all seen data points. And there is no notion about "over-fitting", which means if the model can perfectly pass though all the data points, the model is perfect.

  • On the other hand, "function approximation" may be more used in "machine learning" community, where just like any other learning problems, there are samples from the function (training data), and there are "ground truth function" or "hold out data for validation", therefore we may need to consider the "over-fitting". Where, perfectly predict seen data may not be good enough.

  • $\begingroup$ +1. In ML we have a test set so we always want the expected fit to be good. See the slightly related stats.stackexchange.com/questions/231832/… Essentially we're usually "curve fitting" parameters to some other representation of the target value. Think binary classification were we're predicting the bernoulli parameter, not the target value itself. $\endgroup$
    – ragulpr
    Oct 18 '16 at 14:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.