Determining best fitting curve fitting function out of linear, exponential, and logarithmic functions Context:
From a question on Mathematics Stack Exchange (Can I build a program), someone has a set of $x-y$ points, and wants to fit a curve to it, linear, exponential or logarithmic. 
The usual method is to start by choosing one of these (which specifies the model), and then do the statistical calculations.
But what is really wanted is to find the 'best' curve out of linear, exponential or logarithmic.
Ostensibly, one could try all three, and choose the best fitted curve of the three according to the best correlation coefficient.
But somehow I'm feeling this is not quite kosher. The generally accepted method is to pick your model first, one of those three (or some other link function), then from the data calculate the coefficients. And post facto picking the best of all is cherry picking. But to me whether you're determining a function or coefficients from the data it is still the same thing, your procedure is discovering the best...thing (let's say that which function  is -also- another coefficient o be discovered).
Questions:


*

*Is it appropriate to choose the best fitting model out of linear, exponential, and logarithmic models, based on a comparison of fit statistics?

*If so, what is the most appropriate way to do this?

*If regression helps find parameters (coefficients) in a function, why can't there be a discrete parameter to choose which of three curve families the best would come from?

 A: This is a question that is valid in very diverse domains.
The best model is the one that can predict data points that were not used during the parameter estimation. Ideally one would compute model parameters with a subset of the data set, and evaluate the fit performance on another data set. If you are interested in the details make a search with "cross-validation".
So the answer to first question, is "No". You cannot simply take the best fitting model. Image you are fitting a polynomial with Nth degree to N data points. This will be a perfect fit, because all the model will exactly pass on all data points. However this model will not generalize to new data.
When you do not have enough data to go through the cross-validation procedure in a sound manner, then you can use metrics such as AIC or BIC. These metrics punishes simultaneously the amplitude of residuals and the number of parameters in your model but makes strong assumptions on the generative processes of your data. As these metrics are sensitive to over-fitting, they can be used as a proxy for model selection.
A: Since plenty of people routinely explore the fit of various curves to their data, I don't know where your reservations are coming from.  Granted, there is the fact that a quadratic will always fit at least as well as a linear, and a cubic, at least as well as a quadratic, so there are ways to test the statistical significance of adding such a nonlinear term and thus to avoid needless complexity.  But the basic practice of testing many different forms of a relationship is just good practice.  In fact, one might start with a very flexible loess regression to see what is the most plausible kind of curve to fit. 
A: You really need to find a balance between the science/theory that leads to the data and what the data tells you.  Like others have said, if you let yourself fit any possible transformation (polynomials of any degree, etc.) then you will end up overfitting and getting something that is useless.
One way to convince yourself of this is through simulation.  Choose one of the models (linear, exponential, log) and generate data that follows this model (with a choice of the parameters).  If your conditional variance of the y values is small relative to the spread of the x variable then a simple plot will make it obvious which model was chosen and what the "truth" is.  But if you choose a set of parameters such that it is not obvious from the plots (probably the case where an analytic solution is of interest) then analyze each of the 3 ways and see which gives the "best" fit.  I expect that you will find that the "best" fit is often not the "true" fit.
On the other hand, sometimes we want the data to tell us as much as possible and we may not have the science/theory to fully determine the nature of the relationship.  The original paper by Box and Cox (JRSS B, vol. 26, no. 2, 1964) discusses ways to compare between several transformations on the y variable, their given set of transformations have linear and log as special cases (but not exponential), but nothing in the theory of the paper limits you to only their family of transformations, the same methodology could be extended to include a comparison between the 3 models that you are interested in.
A: *

*You might want to check out the free software called Eureqa. It has the specific aim of automating the process of finding both the functional form and the parameters of a given functional relationship.

*If you are comparing models, with different numbers of parameters, you will generally want to use a measure of fit that penalises models with more parameters. There is a rich literature on which fit measure is most appropriate for model comparison, and issues get more complicated when the models are not nested. I'd be interested to hear what others think is the most suitable model comparison index given your scenario (as a side point, there was recently a discussion on my blog about model comparison indices in the context of comparing models for curve fitting).

*From my experience, non-linear regression models are used for reasons beyond pure statistical fit to the given data: 

*

*Non-linear models make more plausible predictions outside the range of the data

*Non-linear models require fewer parameters for equivalent fit

*Non-linear regression models are often applied in domains where there is substantial prior research and theory guiding model selection.


